Sep 11 19:16:12.970736 ip-10-0-135-147 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 19:16:12.970872 ip-10-0-135-147 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 19:16:12.970965 ip-10-0-135-147 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 19:16:12.971490 ip-10-0-135-147 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 19:16:23.016821 ip-10-0-135-147 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 19:16:23.016839 ip-10-0-135-147 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 0bee1bc92b5c4d15bee979db7f5ed14d --
Sep 11 19:18:14.794672 ip-10-0-135-147 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 19:18:15.199093 ip-10-0-135-147 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 19:18:15.199093 ip-10-0-135-147 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 19:18:15.199093 ip-10-0-135-147 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 19:18:15.199093 ip-10-0-135-147 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 19:18:15.199093 ip-10-0-135-147 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 19:18:15.199941 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.199171    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 19:18:15.202983 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.202962    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 19:18:15.202983 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.202982    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 19:18:15.202983 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.202986    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 19:18:15.202983 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.202991    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.202995    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203002    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203008    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203013    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203017    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203020    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203024    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203028    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203032    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203036    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203040    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203044    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203049    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203052    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203056    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203060    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203064    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203067    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203071    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 19:18:15.203226 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203075    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203079    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203083    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203087    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203091    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203095    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203099    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203104    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203107    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203111    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203117    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203121    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203127    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203131    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203136    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203140    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203144    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203148    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203152    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 19:18:15.204062 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203157    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203161    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203165    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203169    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203174    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203180    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203186    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203190    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203195    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203199    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203203    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203207    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203211    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203215    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203219    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203223    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203227    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203230    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203234    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 19:18:15.204854 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203241    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203246    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203251    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203256    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203260    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203264    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203269    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203274    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203279    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203299    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203304    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203309    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203314    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203318    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203322    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203326    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203330    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203334    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203339    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203344    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 19:18:15.205498 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203348    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203352    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203356    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203360    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203367    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203372    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203376    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203381    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203385    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203389    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203393    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203398    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203402    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203406    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203410    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203414    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203418    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.203422    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204572    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204581    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204587    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 19:18:15.206354 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204592    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204596    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204600    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204604    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204608    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204613    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204617    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204622    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204626    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204630    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204635    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204639    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204643    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204647    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204651    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204655    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204659    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204663    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204668    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 19:18:15.207201 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204672    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204676    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204680    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204684    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204688    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204692    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204696    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204700    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.204705    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205920    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205937    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205943    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205949    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205959    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205966    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205971    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205976    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205982    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205987    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 19:18:15.208012 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205992    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.205997    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206008    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206013    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206019    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206024    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206028    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206033    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206037    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206042    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206049    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206055    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206060    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206065    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206074    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206080    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206084    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206088    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206092    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206099    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 19:18:15.208526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206104    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206109    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206113    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206118    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206122    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206128    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206138    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206145    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206149    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206154    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206158    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206162    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206167    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206171    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206175    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206180    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206185    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206198    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206205    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206210    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 19:18:15.209019 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206215    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206220    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206225    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206230    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206234    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206239    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206244    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206249    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206253    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206257    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206267    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206272    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206277    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206282    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206305    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206310    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206315    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.206319    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207491    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207511    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 19:18:15.209526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207517    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207522    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207527    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207532    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207537    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207542    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207547    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207551    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207556    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207560    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207564    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207573    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207577    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207582    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207586    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207591    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207606    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207612    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207618    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 19:18:15.210004 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207622    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207626    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207630    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207635    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207644    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207650    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207654    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207658    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207662    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207667    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207671    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207675    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207680    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207693    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207698    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207702    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207706    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207714    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207718    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207723    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 19:18:15.210480 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207730    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207734    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207739    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207743    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207747    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207751    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207756    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207760    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207769    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207774    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207779    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207783    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207787    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207792    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207798    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207805    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207811    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207816    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207821    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 19:18:15.210977 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207826    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207830    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207839    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207844    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207848    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207853    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207857    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207869    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.207934    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208083    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208089    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208093    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208096    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208099    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208102    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208105    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208107    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208110    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208113    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208116    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 19:18:15.211484 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208118    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208121    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208123    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208126    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208129    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208132    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208134    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208137    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208139    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208142    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208144    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208147    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208149    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208152    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208155    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208158    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208160    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208163    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208165    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208252    2583 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208303    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 19:18:15.211974 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208311    2583 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208315    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208320    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208323    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208328    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208332    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208336    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208339    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208342    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208346    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208349    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208352    2583 flags.go:64] FLAG: --cgroup-root=""
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208354    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208357    2583 flags.go:64] FLAG: --client-ca-file=""
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208360    2583 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208363    2583 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208368    2583 flags.go:64] FLAG: --cluster-domain=""
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208370    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208373    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208376    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208380    2583 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208384    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208387    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 19:18:15.212539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208390    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208393    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208396    2583 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208399    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208402    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208406    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208408    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208417    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208423    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208427    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208430    2583 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208432    2583 flags.go:64] FLAG: --enable-server="true"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208435    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208440    2583 flags.go:64] FLAG: --event-burst="100"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208443    2583 flags.go:64] FLAG: --event-qps="50"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208446    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208449    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208452    2583 flags.go:64] FLAG: --eviction-hard=""
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208455    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208458    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208461    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208463    2583 flags.go:64] FLAG: --eviction-soft=""
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208467    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208470    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208472    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 19:18:15.213084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208475    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208478    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208481    2583 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208483    2583 flags.go:64] FLAG: --feature-gates=""
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208487    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208490    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208493    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208496    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208499    2583 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208502    2583 flags.go:64] FLAG: --help="false"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208505    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208508    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208511    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208513    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208516    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208521    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208525    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208528    2583 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208531    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208534    2583 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208537    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208540    2583 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208543    2583 flags.go:64] FLAG: --kube-reserved=""
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208546    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 19:18:15.213714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208549    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208552    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208554    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208557    2583 flags.go:64] FLAG: --lock-file=""
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208560    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208562    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208565    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208571    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208573    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208580    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208583    2583 flags.go:64] FLAG: --logging-format="text"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208586    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208589    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208592    2583 flags.go:64] FLAG: --manifest-url=""
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208595    2583 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208599    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208602    2583 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208606    2583 flags.go:64] FLAG: --max-pods="110"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208609    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208612    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208615    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208617    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208620    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208625    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208629    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 19:18:15.214312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208636    2583 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208640    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208643    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208646    2583 flags.go:64] FLAG: --pod-cidr=""
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208649    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208656    2583 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208659    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208662    2583 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208664    2583 flags.go:64] FLAG: --port="10250"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208667    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208670    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0877fb98ff3140341"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208673    2583 flags.go:64] FLAG: --qos-reserved=""
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208676    2583 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208679    2583 flags.go:64] FLAG: --register-node="true"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208682    2583 flags.go:64] FLAG: --register-with-taints=""
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208685    2583 flags.go:64] FLAG: --registry-burst="10"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208688    2583 flags.go:64] FLAG: --registry-qps="5"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208690    2583 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208694    2583 flags.go:64] FLAG: --reserved-memory=""
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208697    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208700    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208703    2583 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208706    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208709    2583 flags.go:64] FLAG: --runonce="false"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208711    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 19:18:15.214968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208714    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208717    2583 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208720    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208723    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208726    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208729    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208732    2583 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208737    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208739    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208742    2583 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208745    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208748    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208751    2583 flags.go:64] FLAG: --system-cgroups=""
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208753    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208759    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208761    2583 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208764    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208768    2583 flags.go:64] FLAG: --tls-min-version=""
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208772    2583 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208774    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208777    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208780    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208789    2583 flags.go:64] FLAG: --v="2"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208793    2583 flags.go:64] FLAG: --version="false"
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208797    2583 flags.go:64] FLAG: --vmodule=""
Sep 11 19:18:15.215592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208800    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.208803    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208926    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208930    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208933    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208936    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208939    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208942    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208945    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208947    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208950    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208952    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208956    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208959    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208963    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208965    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208968    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208971    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208973    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208976    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208979    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 19:18:15.216206 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208981    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208984    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208986    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208989    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208991    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208994    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208996    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.208999    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209001    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209009    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209012    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209016    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209020    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209023    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209026    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209029    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209031    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209035    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209039    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 19:18:15.216753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209042    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209045    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209047    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209050    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209054    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209056    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209073    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209075    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209078    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209081    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209083    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209086    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209089    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209091    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209094    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209096    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209099    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209101    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209104    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 19:18:15.217208 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209106    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209109    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209111    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209114    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209124    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209126    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209129    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209131    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209134    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209137    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209139    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209142    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209144    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209146    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209149    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209151    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209154    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209157    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209160    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209164    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 19:18:15.217693 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209166    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209169    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209172    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209174    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209177    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209180    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209184    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209187    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209190    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209193    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209196    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209199    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209201    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209204    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209206    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209209    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209211    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209220    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209223    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209225    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 19:18:15.218183 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209227    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.209230    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.209715    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.216791    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.216806    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.217498    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.217510    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217571    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217576    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217580    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217583    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217586    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217589    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217592    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217595    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 19:18:15.218734 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217597    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217600    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217603    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217606    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217608    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217611    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217614    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217616    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217619    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217621    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217624    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217626    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217629    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217632    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217640    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217643    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217645    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217648    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217652    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 19:18:15.219105 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217657    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217659    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217663    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217666    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217669    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217672    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217674    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217677    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217680    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217682    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217685    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217688    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217690    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217692    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217695    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217698    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217700    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217703    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217705    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217708    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 19:18:15.219591 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217711    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217713    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217715    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217718    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217720    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217723    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217725    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217728    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217732    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217735    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217738    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217740    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217743    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217745    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217749    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217751    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217754    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217756    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217759    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217761    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 19:18:15.220082 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217764    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217767    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217771    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217774    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217777    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217779    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217782    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217784    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217787    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217790    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217792    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217795    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217797    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217800    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217802    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217805    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217807    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217810    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217813    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 19:18:15.220590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217817    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217820    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217824    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217827    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217830    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217832    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217835    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217837    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217840    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217843    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217846    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217848    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217851    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.217856    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217987    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 19:18:15.221039 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217993    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217996    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.217999    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218002    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218004    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218007    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218010    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218012    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218015    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218017    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218020    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218022    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218024    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218027    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218029    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218032    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218034    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218037    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218039    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218043    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 19:18:15.221432 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218050    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218053    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218056    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218059    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218062    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218064    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218067    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218069    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218072    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218074    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218077    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218079    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218082    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218084    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218087    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218089    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218092    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218095    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218098    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218100    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 19:18:15.221911 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218103    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218105    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218108    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218110    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218113    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218116    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218118    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218121    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218123    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218127    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218129    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218131    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218134    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218137    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218140    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218142    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218145    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218147    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218150    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 19:18:15.222420 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218153    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218155    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218158    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218160    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218162    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218165    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218167    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218170    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218172    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218175    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218177    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218180    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218182    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218185    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218188    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218190    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218193    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218195    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218198    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218200    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218203    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 19:18:15.222871 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218205    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218208    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218210    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218213    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218215    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218218    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218221    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218223    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218226    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218228    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218231    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218233    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218235    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218238    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218240    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218244    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218247    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 19:18:15.223393 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:15.218250    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 19:18:15.223793 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.218255    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 19:18:15.223793 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.218389    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 19:18:15.223793 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.220370    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 19:18:15.223793 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.221322    2583 server.go:1019] "Starting client certificate rotation"
Sep 11 19:18:15.223793 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.221422    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 19:18:15.223793 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.221462    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 19:18:15.223986 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.223931    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 19:18:15.229092 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.229072    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 19:18:15.238436 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.238417    2583 log.go:25] "Validated CRI v1 runtime API"
Sep 11 19:18:15.241423 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.241406    2583 log.go:25] "Validated CRI v1 image API"
Sep 11 19:18:15.243326 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.243307    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 19:18:15.246212 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.246192    2583 fs.go:135] Filesystem UUIDs: map[1b7b19f6-df7d-4999-8a26-cae3b0dfea74:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 eb79e27f-fa5f-479c-83a1-eba62fd9a11f:/dev/nvme0n1p3]
Sep 11 19:18:15.246270 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.246213    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 19:18:15.252326 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.252309    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 19:18:15.252420 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.252276    2583 manager.go:216] Machine: {Timestamp:2026-09-11 19:18:15.250909629 +0000 UTC m=+0.341486709 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099784 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2be08abb50c2ebaba4ea63ffca262f SystemUUID:ec2be08a-bb50-c2eb-aba4-ea63ffca262f BootID:0bee1bc9-2b5c-4d15-bee9-79db7f5ed14d Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1d:3d:ab:9b:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1d:3d:ab:9b:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:c7:fc:cb:fd:ec Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 19:18:15.252420 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.252399    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 19:18:15.252583 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.252486    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 19:18:15.255169 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.255144    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 19:18:15.255328 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.255171    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-147.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 19:18:15.256398 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.256387    2583 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 19:18:15.256438 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.256400    2583 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 19:18:15.256438 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.256409    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 19:18:15.256991 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.256981    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 19:18:15.257714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.257704    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 19:18:15.260894 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.260883    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 19:18:15.261005 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.260996    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 19:18:15.261043 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.261027    2583 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 19:18:15.261043 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.261037    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 19:18:15.261093 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.261048    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 19:18:15.261093 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.261057    2583 kubelet.go:398] "Adding apiserver pod source"
Sep 11 19:18:15.261093 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.261067    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 19:18:15.261501 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.261488    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 19:18:15.261573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.261509    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 19:18:15.263915 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.263900    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 19:18:15.265811 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.265798    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 19:18:15.265863 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.265829    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 19:18:15.267519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267502    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 19:18:15.267519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267518    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 19:18:15.267519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267524    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 19:18:15.267738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267529    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 19:18:15.267738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267552    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 19:18:15.267738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267557    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 19:18:15.267738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267572    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 19:18:15.267738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267577    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 19:18:15.267738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267584    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 19:18:15.267738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267606    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 19:18:15.267738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267622    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 19:18:15.267738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.267629    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 19:18:15.268396 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.268385    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 19:18:15.268396 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.268394    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 19:18:15.271245 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.271221    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-147.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 19:18:15.271365 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.271258    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-147.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 19:18:15.271365 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.271282    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 19:18:15.272262 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.272246    2583 server.go:1268] "Started kubelet"
Sep 11 19:18:15.272495 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.272432    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 19:18:15.272596 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.272425    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 19:18:15.272596 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.272542    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 19:18:15.273163 ip-10-0-135-147 systemd[1]: Started Kubernetes Kubelet.
Sep 11 19:18:15.276622 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.276589    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ksztw"
Sep 11 19:18:15.277171 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.277131    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 19:18:15.277776 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.277756    2583 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 19:18:15.278115 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.278096    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 19:18:15.280860 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.280075    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-147.ec2.internal.18d45a5a2a10bc56  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-147.ec2.internal,UID:ip-10-0-135-147.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-147.ec2.internal,},FirstTimestamp:2026-09-11 19:18:15.272225878 +0000 UTC m=+0.362802960,LastTimestamp:2026-09-11 19:18:15.272225878 +0000 UTC m=+0.362802960,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-147.ec2.internal,}"
Sep 11 19:18:15.281460 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.281441    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 19:18:15.282569 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.282552    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 19:18:15.282674 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.282653    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:15.283173 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283150    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 19:18:15.283173 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283175    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 19:18:15.283331 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283154    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 19:18:15.283331 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283308    2583 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 19:18:15.283331 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283317    2583 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 19:18:15.283538 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283520    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 19:18:15.283608 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283540    2583 factory.go:55] Registering systemd factory
Sep 11 19:18:15.283608 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283549    2583 factory.go:223] Registration of the systemd container factory successfully
Sep 11 19:18:15.283753 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283738    2583 factory.go:153] Registering CRI-O factory
Sep 11 19:18:15.283809 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283755    2583 factory.go:223] Registration of the crio container factory successfully
Sep 11 19:18:15.283809 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283769    2583 factory.go:103] Registering Raw factory
Sep 11 19:18:15.283809 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283779    2583 manager.go:1196] Started watching for new ooms in manager
Sep 11 19:18:15.283939 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.283918    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ksztw"
Sep 11 19:18:15.284120 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.284109    2583 manager.go:349] Starting recovery of all containers
Sep 11 19:18:15.293145 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.293114    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 19:18:15.293688 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.293672    2583 manager.go:354] Recovery completed
Sep 11 19:18:15.295508 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.295482    2583 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-147.ec2.internal\" not found" node="ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.298479 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.298467    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 19:18:15.300128 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.300114    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 19:18:15.300179 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.300148    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 19:18:15.300179 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.300165    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientPID"
Sep 11 19:18:15.300707 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.300691    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 19:18:15.300707 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.300704    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 19:18:15.300820 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.300720    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 19:18:15.300851 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.300705    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 19:18:15.301583 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.301570    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 19:18:15.301635 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.301596    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 19:18:15.301635 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.301608    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientPID"
Sep 11 19:18:15.301805 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.301791    2583 policy_none.go:49] "None policy: Start"
Sep 11 19:18:15.301837 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.301807    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 19:18:15.301837 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.301816    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 19:18:15.302827 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.302817    2583 policy_none.go:47] "Start"
Sep 11 19:18:15.348188 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.347870    2583 manager.go:337] "Starting Device Plugin manager"
Sep 11 19:18:15.348188 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.347917    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 19:18:15.348188 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.347931    2583 server.go:87] "Starting device plugin registration server"
Sep 11 19:18:15.348439 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.348235    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 19:18:15.348439 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.348248    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 19:18:15.348439 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.348356    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 19:18:15.348573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.348447    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 19:18:15.348573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.348456    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 19:18:15.349048 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.349026    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 19:18:15.349152 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.349077    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:15.376276 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.376241    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 19:18:15.377636 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.377619    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 19:18:15.377728 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.377676    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 19:18:15.377728 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.377710    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 19:18:15.377817 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.377758    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 19:18:15.379786 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.379767    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 19:18:15.449150 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.449066    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 19:18:15.450050 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.450033    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 19:18:15.450156 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.450067    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 19:18:15.450156 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.450082    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientPID"
Sep 11 19:18:15.450156 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.450115    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.458247 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.458229    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.458369 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.458258    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-147.ec2.internal\": node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:15.478595 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.478567    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-147.ec2.internal"]
Sep 11 19:18:15.478723 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.478636    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 19:18:15.480032 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.480018    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 19:18:15.480102 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.480047    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 19:18:15.480102 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.480056    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientPID"
Sep 11 19:18:15.480505 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.480486    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:15.481183 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.481170    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 19:18:15.481312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.481277    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.481353 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.481323    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 19:18:15.481860 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.481843    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 19:18:15.481860 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.481858    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 19:18:15.481979 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.481880    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 19:18:15.481979 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.481880    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 19:18:15.481979 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.481891    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientPID"
Sep 11 19:18:15.481979 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.481898    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientPID"
Sep 11 19:18:15.483046 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.483033    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.483090 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.483058    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 19:18:15.483788 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.483770    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 19:18:15.483867 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.483803    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 19:18:15.483867 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.483818    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeHasSufficientPID"
Sep 11 19:18:15.483945 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.483876    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/64cf4cd8df0a6677bea382d1c5122f9a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-147.ec2.internal\" (UID: \"64cf4cd8df0a6677bea382d1c5122f9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.483945 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.483899    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6416ebedd89de9f7c7c06d3da5392139-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal\" (UID: \"6416ebedd89de9f7c7c06d3da5392139\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.483945 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.483938    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6416ebedd89de9f7c7c06d3da5392139-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal\" (UID: \"6416ebedd89de9f7c7c06d3da5392139\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.505159 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.505128    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-147.ec2.internal\" not found" node="ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.509422 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.509405    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-147.ec2.internal\" not found" node="ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.581630 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.581597    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:15.584851 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.584829    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6416ebedd89de9f7c7c06d3da5392139-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal\" (UID: \"6416ebedd89de9f7c7c06d3da5392139\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.584910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.584865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6416ebedd89de9f7c7c06d3da5392139-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal\" (UID: \"6416ebedd89de9f7c7c06d3da5392139\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.584910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.584884    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/64cf4cd8df0a6677bea382d1c5122f9a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-147.ec2.internal\" (UID: \"64cf4cd8df0a6677bea382d1c5122f9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.584978 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.584927    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/64cf4cd8df0a6677bea382d1c5122f9a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-147.ec2.internal\" (UID: \"64cf4cd8df0a6677bea382d1c5122f9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.584978 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.584928    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6416ebedd89de9f7c7c06d3da5392139-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal\" (UID: \"6416ebedd89de9f7c7c06d3da5392139\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.584978 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.584930    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6416ebedd89de9f7c7c06d3da5392139-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal\" (UID: \"6416ebedd89de9f7c7c06d3da5392139\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.681766 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.681731    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:15.782539 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.782474    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:15.807900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.807858    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.812791 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:15.812765    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:15.883488 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.883301    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:15.983679 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:15.983646    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:16.084170 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:16.084090    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:16.184518 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:16.184486    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:16.220832 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:16.220804    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 19:18:16.221268 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:16.220963    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 19:18:16.221268 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:16.220991    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 19:18:16.284758 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:16.284736    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:16.287391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:16.287367    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 19:13:16 +0000 UTC" deadline="2028-06-21 01:08:25.482478686 +0000 UTC"
Sep 11 19:18:16.287391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:16.287391    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15557h50m9.195091581s"
Sep 11 19:18:16.290074 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:16.290050    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6416ebedd89de9f7c7c06d3da5392139.slice/crio-3a8582825bb4cde4cc26c5c3fefc2ab0440d2fb54b6175e671991e2f70c5c1da WatchSource:0}: Error finding container 3a8582825bb4cde4cc26c5c3fefc2ab0440d2fb54b6175e671991e2f70c5c1da: Status 404 returned error can't find the container with id 3a8582825bb4cde4cc26c5c3fefc2ab0440d2fb54b6175e671991e2f70c5c1da
Sep 11 19:18:16.290339 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:16.290321    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64cf4cd8df0a6677bea382d1c5122f9a.slice/crio-e88c6c229f9dd425850ffd3da3c07761d183753b476975ed148688769fbcc253 WatchSource:0}: Error finding container e88c6c229f9dd425850ffd3da3c07761d183753b476975ed148688769fbcc253: Status 404 returned error can't find the container with id e88c6c229f9dd425850ffd3da3c07761d183753b476975ed148688769fbcc253
Sep 11 19:18:16.296317 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:16.296301    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 19:18:16.381433 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:16.381328    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal" event={"ID":"6416ebedd89de9f7c7c06d3da5392139","Type":"ContainerStarted","Data":"3a8582825bb4cde4cc26c5c3fefc2ab0440d2fb54b6175e671991e2f70c5c1da"}
Sep 11 19:18:16.382410 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:16.382388    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-147.ec2.internal" event={"ID":"64cf4cd8df0a6677bea382d1c5122f9a","Type":"ContainerStarted","Data":"e88c6c229f9dd425850ffd3da3c07761d183753b476975ed148688769fbcc253"}
Sep 11 19:18:16.385610 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:16.385595    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:16.486174 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:16.486138    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:16.586667 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:16.586631    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:17.789930 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:17.789906    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 19:18:17.849122 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:17.849052    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:17.949846 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:17.949808    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-147.ec2.internal\" not found"
Sep 11 19:18:17.969268 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:17.969050    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 19:18:18.044987 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.044779    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:18.055968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.055940    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 19:18:18.057191 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.056957    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal"
Sep 11 19:18:18.065961 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.065937    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 19:18:18.424973 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.424939    2583 apiserver.go:52] "Watching apiserver"
Sep 11 19:18:18.430380 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.430357    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 19:18:18.430777 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.430747    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal","openshift-multus/multus-additional-cni-plugins-xxfp8","kube-system/global-pull-secret-syncer-69wfq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9","openshift-image-registry/node-ca-277dr","openshift-multus/multus-km429","openshift-multus/network-metrics-daemon-dvg7l","openshift-network-diagnostics/network-check-target-m6tdc","openshift-network-operator/iptables-alerter-76dkk","openshift-ovn-kubernetes/ovnkube-node-kfhvg","kube-system/konnectivity-agent-cctlc","kube-system/kube-apiserver-proxy-ip-10-0-135-147.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qhppv","openshift-dns/node-resolver-7hsvz"]
Sep 11 19:18:18.432213 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.432193    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:18.432320 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:18.432280    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:18.433266 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.433231    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.434323 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.434300    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:18.434415 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:18.434378    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:18.436966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.435346    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.436966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.435493    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 19:18:18.436966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.435544    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 19:18:18.436966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.435766    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gd6md\""
Sep 11 19:18:18.436966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.436383    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-277dr"
Sep 11 19:18:18.437438 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.437094    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 19:18:18.437438 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.437183    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 19:18:18.437656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.437638    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 19:18:18.437720 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.437706    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nm75x\""
Sep 11 19:18:18.437777 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.437757    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-km429"
Sep 11 19:18:18.438348 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.438163    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 19:18:18.438348 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.438306    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 19:18:18.438348 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.438320    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 19:18:18.438538 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.438493    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7pk2m\""
Sep 11 19:18:18.438706 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.438692    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-76dkk"
Sep 11 19:18:18.439453 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.439436    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 19:18:18.439802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.439786    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 19:18:18.440067 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.440052    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 19:18:18.440136 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.440076    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5lz2q\""
Sep 11 19:18:18.440317 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.440302    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7hsvz"
Sep 11 19:18:18.440433 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.440416    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 19:18:18.440651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.440637    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fg9sr\""
Sep 11 19:18:18.441111 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.440838    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 19:18:18.441152 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.441116    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 19:18:18.441152 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.441122    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 19:18:18.441565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.441542    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.442436 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.442410    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 19:18:18.442552 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.442415    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 19:18:18.442990 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.442943    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qncxq\""
Sep 11 19:18:18.443456 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.443434    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.444753 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.444732    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5hcl8\""
Sep 11 19:18:18.445018 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.444990    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 19:18:18.445586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.445567    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 19:18:18.446024 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.445996    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rznzr\""
Sep 11 19:18:18.446503 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.446484    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 19:18:18.446661 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.446644    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 19:18:18.446744 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.446710    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 19:18:18.447148 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.447129    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 19:18:18.447945 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.447306    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 19:18:18.447945 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.447512    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 19:18:18.448145 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.448122    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:18.448648 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.448627    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 19:18:18.449404 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.449386    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:18.449507 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:18.449451    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:18.450223 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.450200    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 19:18:18.450692 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.450348    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-875v9\""
Sep 11 19:18:18.450692 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.450611    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 19:18:18.455384 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455361    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc814053-fb3d-45bb-bc8e-ea849d8f1951-cnibin\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.455477 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455397    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-system-cni-dir\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.455477 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455422    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-systemd\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.455477 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455444    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-host\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.455477 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455468    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7dac3817-ba81-4281-a22f-44ebd2700b4c-iptables-alerter-script\") pod \"iptables-alerter-76dkk\" (UID: \"7dac3817-ba81-4281-a22f-44ebd2700b4c\") " pod="openshift-network-operator/iptables-alerter-76dkk"
Sep 11 19:18:18.455649 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455519    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc814053-fb3d-45bb-bc8e-ea849d8f1951-cni-binary-copy\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.455649 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cc814053-fb3d-45bb-bc8e-ea849d8f1951-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.455649 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455615    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-run-multus-certs\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.455649 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455640    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk6h5\" (UniqueName: \"kubernetes.io/projected/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-kube-api-access-qk6h5\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.455841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455665    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.455841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455691    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:18.455841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455725    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc814053-fb3d-45bb-bc8e-ea849d8f1951-system-cni-dir\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.455841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455755    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-var-lib-openvswitch\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.455841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455795    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd636fcf-c98a-4d32-a54f-10aa4648580b-agent-certs\") pod \"konnectivity-agent-cctlc\" (UID: \"bd636fcf-c98a-4d32-a54f-10aa4648580b\") " pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:18.455841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455816    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/baaf76a2-be45-4361-9e59-d750e56eba21-etc-tuned\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.455841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455831    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-hostroot\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455856    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkpm7\" (UniqueName: \"kubernetes.io/projected/46a71960-5367-47ee-9326-60be4aeea347-kube-api-access-vkpm7\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455869    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6-serviceca\") pod \"node-ca-277dr\" (UID: \"cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6\") " pod="openshift-image-registry/node-ca-277dr"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455887    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-cni-binary-copy\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455909    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-run-netns\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455925    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-systemd-units\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455942    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-slash\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455958    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m84lq\" (UniqueName: \"kubernetes.io/projected/96f8ca15-1313-46e5-bcfb-9639b2736b7b-kube-api-access-m84lq\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.455973    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd636fcf-c98a-4d32-a54f-10aa4648580b-konnectivity-ca\") pod \"konnectivity-agent-cctlc\" (UID: \"bd636fcf-c98a-4d32-a54f-10aa4648580b\") " pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456007    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-kubernetes\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456041    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt5vj\" (UniqueName: \"kubernetes.io/projected/baaf76a2-be45-4361-9e59-d750e56eba21-kube-api-access-vt5vj\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456092    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nsdf\" (UniqueName: \"kubernetes.io/projected/cc814053-fb3d-45bb-bc8e-ea849d8f1951-kube-api-access-7nsdf\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.456153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456138    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw8jb\" (UniqueName: \"kubernetes.io/projected/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-kube-api-access-sw8jb\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456171    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-run-openvswitch\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6-host\") pod \"node-ca-277dr\" (UID: \"cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6\") " pod="openshift-image-registry/node-ca-277dr"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456223    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3754f687-4a22-4dbe-9151-d33ccbc0afd5-hosts-file\") pod \"node-resolver-7hsvz\" (UID: \"3754f687-4a22-4dbe-9151-d33ccbc0afd5\") " pod="openshift-dns/node-resolver-7hsvz"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456261    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-node-log\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456292    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-kubelet-config\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456318    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-sys-fs\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456343    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456372    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-var-lib-cni-multus\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456394    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-var-lib-kubelet\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456417    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7dac3817-ba81-4281-a22f-44ebd2700b4c-host-slash\") pod \"iptables-alerter-76dkk\" (UID: \"7dac3817-ba81-4281-a22f-44ebd2700b4c\") " pod="openshift-network-operator/iptables-alerter-76dkk"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456439    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqzt7\" (UniqueName: \"kubernetes.io/projected/7dac3817-ba81-4281-a22f-44ebd2700b4c-kube-api-access-cqzt7\") pod \"iptables-alerter-76dkk\" (UID: \"7dac3817-ba81-4281-a22f-44ebd2700b4c\") " pod="openshift-network-operator/iptables-alerter-76dkk"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-run-ovn-kubernetes\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456526    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-etc-selinux\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc814053-fb3d-45bb-bc8e-ea849d8f1951-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-cnibin\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.456677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456628    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-cni-bin\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456643    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/96f8ca15-1313-46e5-bcfb-9639b2736b7b-env-overrides\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456670    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-sysctl-conf\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456686    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-registration-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456706    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-device-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456730    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-etc-kubernetes\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456754    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-cni-netd\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/96f8ca15-1313-46e5-bcfb-9639b2736b7b-ovnkube-config\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/96f8ca15-1313-46e5-bcfb-9639b2736b7b-ovn-node-metrics-cert\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456836    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/96f8ca15-1313-46e5-bcfb-9639b2736b7b-ovnkube-script-lib\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456851    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-modprobe-d\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456864    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-sys\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456883    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-lib-modules\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456933    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-conf-dir\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456960    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fsxl\" (UniqueName: \"kubernetes.io/projected/3754f687-4a22-4dbe-9151-d33ccbc0afd5-kube-api-access-6fsxl\") pod \"node-resolver-7hsvz\" (UID: \"3754f687-4a22-4dbe-9151-d33ccbc0afd5\") " pod="openshift-dns/node-resolver-7hsvz"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.456998    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-run-netns\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.457354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457023    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-sysconfig\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457044    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-cni-dir\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457065    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-run-k8s-cni-cncf-io\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457096    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-daemon-config\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457119    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-log-socket\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457135    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-run-systemd\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457155    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-sysctl-d\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457177    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/baaf76a2-be45-4361-9e59-d750e56eba21-tmp\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-socket-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457233    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-var-lib-cni-bin\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457281    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-var-lib-kubelet\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457273    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457304    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3754f687-4a22-4dbe-9151-d33ccbc0afd5-tmp-dir\") pod \"node-resolver-7hsvz\" (UID: \"3754f687-4a22-4dbe-9151-d33ccbc0afd5\") " pod="openshift-dns/node-resolver-7hsvz"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457334    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-kubelet\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457353    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-dbus\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457378    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457399    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cc814053-fb3d-45bb-bc8e-ea849d8f1951-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.458108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457442    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-socket-dir-parent\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.458780 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-etc-openvswitch\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.458780 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457497    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-run-ovn\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.458780 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457518    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-run\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.458780 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.457560    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn25z\" (UniqueName: \"kubernetes.io/projected/cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6-kube-api-access-mn25z\") pod \"node-ca-277dr\" (UID: \"cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6\") " pod="openshift-image-registry/node-ca-277dr"
Sep 11 19:18:18.465303 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.465286    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 19:18:18.474097 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.474078    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 19:18:18.479096 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.479075    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dp2wj"
Sep 11 19:18:18.485019 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.484759    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dp2wj"
Sep 11 19:18:18.546100 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.546072    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 19:18:18.558536 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558504    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-var-lib-cni-bin\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.558536 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558540    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-var-lib-kubelet\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.558752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558590    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3754f687-4a22-4dbe-9151-d33ccbc0afd5-tmp-dir\") pod \"node-resolver-7hsvz\" (UID: \"3754f687-4a22-4dbe-9151-d33ccbc0afd5\") " pod="openshift-dns/node-resolver-7hsvz"
Sep 11 19:18:18.558752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558593    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-var-lib-cni-bin\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.558752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558608    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-kubelet\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.558752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558607    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-var-lib-kubelet\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.558752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558644    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-dbus\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:18.558752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558661    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-kubelet\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.558752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558672    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.558752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558696    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cc814053-fb3d-45bb-bc8e-ea849d8f1951-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.558752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558726    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-socket-dir-parent\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.558752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558744    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.558752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558750    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-etc-openvswitch\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558774    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-run-ovn\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558800    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-run\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558826    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn25z\" (UniqueName: \"kubernetes.io/projected/cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6-kube-api-access-mn25z\") pod \"node-ca-277dr\" (UID: \"cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6\") " pod="openshift-image-registry/node-ca-277dr"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558872    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-dbus\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558881    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc814053-fb3d-45bb-bc8e-ea849d8f1951-cnibin\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558909    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-system-cni-dir\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558897    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-socket-dir-parent\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558934    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-systemd\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558943    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3754f687-4a22-4dbe-9151-d33ccbc0afd5-tmp-dir\") pod \"node-resolver-7hsvz\" (UID: \"3754f687-4a22-4dbe-9151-d33ccbc0afd5\") " pod="openshift-dns/node-resolver-7hsvz"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558958    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-host\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558964    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc814053-fb3d-45bb-bc8e-ea849d8f1951-cnibin\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558976    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-etc-openvswitch\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.558943    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-run\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559023    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-run-ovn\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559036    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-host\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559081    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-system-cni-dir\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559122    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-systemd\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.559290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559156    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7dac3817-ba81-4281-a22f-44ebd2700b4c-iptables-alerter-script\") pod \"iptables-alerter-76dkk\" (UID: \"7dac3817-ba81-4281-a22f-44ebd2700b4c\") " pod="openshift-network-operator/iptables-alerter-76dkk"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559189    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc814053-fb3d-45bb-bc8e-ea849d8f1951-cni-binary-copy\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559222    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cc814053-fb3d-45bb-bc8e-ea849d8f1951-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559267    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-run-multus-certs\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559292    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk6h5\" (UniqueName: \"kubernetes.io/projected/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-kube-api-access-qk6h5\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559314    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559375    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559405    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s5kl\" (UniqueName: \"kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl\") pod \"network-check-target-m6tdc\" (UID: \"c9378a65-f0bc-4a82-863c-a1647b854dd7\") " pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559430    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc814053-fb3d-45bb-bc8e-ea849d8f1951-system-cni-dir\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559462    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-var-lib-openvswitch\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559488    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd636fcf-c98a-4d32-a54f-10aa4648580b-agent-certs\") pod \"konnectivity-agent-cctlc\" (UID: \"bd636fcf-c98a-4d32-a54f-10aa4648580b\") " pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559514    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/baaf76a2-be45-4361-9e59-d750e56eba21-etc-tuned\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-hostroot\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559554    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7dac3817-ba81-4281-a22f-44ebd2700b4c-iptables-alerter-script\") pod \"iptables-alerter-76dkk\" (UID: \"7dac3817-ba81-4281-a22f-44ebd2700b4c\") " pod="openshift-network-operator/iptables-alerter-76dkk"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559566    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkpm7\" (UniqueName: \"kubernetes.io/projected/46a71960-5367-47ee-9326-60be4aeea347-kube-api-access-vkpm7\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559462    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cc814053-fb3d-45bb-bc8e-ea849d8f1951-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6-serviceca\") pod \"node-ca-277dr\" (UID: \"cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6\") " pod="openshift-image-registry/node-ca-277dr"
Sep 11 19:18:18.560157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559620    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-cni-binary-copy\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:18.559631    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559755    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cc814053-fb3d-45bb-bc8e-ea849d8f1951-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559811    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-var-lib-openvswitch\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559898    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc814053-fb3d-45bb-bc8e-ea849d8f1951-system-cni-dir\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559924    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.559954    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560027    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-run-multus-certs\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560054    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc814053-fb3d-45bb-bc8e-ea849d8f1951-cni-binary-copy\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560107    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-hostroot\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560155    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6-serviceca\") pod \"node-ca-277dr\" (UID: \"cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6\") " pod="openshift-image-registry/node-ca-277dr"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:18.560180    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret podName:c2c8ebda-08bc-4433-9b6d-e4344b72ee3b nodeName:}" failed. No retries permitted until 2026-09-11 19:18:19.060133388 +0000 UTC m=+2.988758283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret") pod "global-pull-secret-syncer-69wfq" (UID: "c2c8ebda-08bc-4433-9b6d-e4344b72ee3b") : object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560210    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-run-netns\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560289    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-systemd-units\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560289    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-run-netns\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560328    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-slash\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560347    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-systemd-units\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.560949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560353    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m84lq\" (UniqueName: \"kubernetes.io/projected/96f8ca15-1313-46e5-bcfb-9639b2736b7b-kube-api-access-m84lq\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560385    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd636fcf-c98a-4d32-a54f-10aa4648580b-konnectivity-ca\") pod \"konnectivity-agent-cctlc\" (UID: \"bd636fcf-c98a-4d32-a54f-10aa4648580b\") " pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560414    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-kubernetes\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560442    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt5vj\" (UniqueName: \"kubernetes.io/projected/baaf76a2-be45-4361-9e59-d750e56eba21-kube-api-access-vt5vj\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560464    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-slash\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nsdf\" (UniqueName: \"kubernetes.io/projected/cc814053-fb3d-45bb-bc8e-ea849d8f1951-kube-api-access-7nsdf\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560494    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw8jb\" (UniqueName: \"kubernetes.io/projected/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-kube-api-access-sw8jb\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560512    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-run-openvswitch\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6-host\") pod \"node-ca-277dr\" (UID: \"cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6\") " pod="openshift-image-registry/node-ca-277dr"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560585    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3754f687-4a22-4dbe-9151-d33ccbc0afd5-hosts-file\") pod \"node-resolver-7hsvz\" (UID: \"3754f687-4a22-4dbe-9151-d33ccbc0afd5\") " pod="openshift-dns/node-resolver-7hsvz"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560707    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-node-log\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560713    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-kubernetes\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560723    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-kubelet-config\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560764    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-kubelet-config\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560823    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6-host\") pod \"node-ca-277dr\" (UID: \"cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6\") " pod="openshift-image-registry/node-ca-277dr"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560872    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-sys-fs\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560897    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-run-openvswitch\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.561726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560889    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560923    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-var-lib-cni-multus\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560944    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3754f687-4a22-4dbe-9151-d33ccbc0afd5-hosts-file\") pod \"node-resolver-7hsvz\" (UID: \"3754f687-4a22-4dbe-9151-d33ccbc0afd5\") " pod="openshift-dns/node-resolver-7hsvz"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560951    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd636fcf-c98a-4d32-a54f-10aa4648580b-konnectivity-ca\") pod \"konnectivity-agent-cctlc\" (UID: \"bd636fcf-c98a-4d32-a54f-10aa4648580b\") " pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560970    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-var-lib-cni-multus\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:18.560985    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560991    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-node-log\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.560998    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-var-lib-kubelet\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:18.561017    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs podName:c1b7247e-b7a4-44db-a5dd-4817bb0df58a nodeName:}" failed. No retries permitted until 2026-09-11 19:18:19.061006619 +0000 UTC m=+2.989631518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs") pod "network-metrics-daemon-dvg7l" (UID: "c1b7247e-b7a4-44db-a5dd-4817bb0df58a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561052    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-sys-fs\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7dac3817-ba81-4281-a22f-44ebd2700b4c-host-slash\") pod \"iptables-alerter-76dkk\" (UID: \"7dac3817-ba81-4281-a22f-44ebd2700b4c\") " pod="openshift-network-operator/iptables-alerter-76dkk"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561099    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7dac3817-ba81-4281-a22f-44ebd2700b4c-host-slash\") pod \"iptables-alerter-76dkk\" (UID: \"7dac3817-ba81-4281-a22f-44ebd2700b4c\") " pod="openshift-network-operator/iptables-alerter-76dkk"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561120    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqzt7\" (UniqueName: \"kubernetes.io/projected/7dac3817-ba81-4281-a22f-44ebd2700b4c-kube-api-access-cqzt7\") pod \"iptables-alerter-76dkk\" (UID: \"7dac3817-ba81-4281-a22f-44ebd2700b4c\") " pod="openshift-network-operator/iptables-alerter-76dkk"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561100    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-var-lib-kubelet\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561196    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-cni-binary-copy\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561154    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-run-ovn-kubernetes\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561265    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-etc-selinux\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.562514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561277    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-run-ovn-kubernetes\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561295    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc814053-fb3d-45bb-bc8e-ea849d8f1951-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561318    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-cnibin\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561358    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-cni-bin\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561361    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-etc-selinux\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561383    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/96f8ca15-1313-46e5-bcfb-9639b2736b7b-env-overrides\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561406    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-sysctl-conf\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561411    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-cni-bin\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561431    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-registration-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561428    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-cnibin\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561445    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc814053-fb3d-45bb-bc8e-ea849d8f1951-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561451    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-device-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561491    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-etc-kubernetes\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561507    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-registration-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561518    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-cni-netd\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561494    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-device-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561536    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-etc-kubernetes\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.563564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561576    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-cni-netd\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561591    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-sysctl-conf\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561605    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/96f8ca15-1313-46e5-bcfb-9639b2736b7b-ovnkube-config\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561644    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/96f8ca15-1313-46e5-bcfb-9639b2736b7b-ovn-node-metrics-cert\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561667    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/96f8ca15-1313-46e5-bcfb-9639b2736b7b-ovnkube-script-lib\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561689    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-modprobe-d\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561710    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-sys\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561745    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-lib-modules\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561771    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-conf-dir\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561796    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fsxl\" (UniqueName: \"kubernetes.io/projected/3754f687-4a22-4dbe-9151-d33ccbc0afd5-kube-api-access-6fsxl\") pod \"node-resolver-7hsvz\" (UID: \"3754f687-4a22-4dbe-9151-d33ccbc0afd5\") " pod="openshift-dns/node-resolver-7hsvz"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561850    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-run-netns\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561854    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/96f8ca15-1313-46e5-bcfb-9639b2736b7b-env-overrides\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561894    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-sysconfig\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561916    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-conf-dir\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561918    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-cni-dir\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561953    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-run-k8s-cni-cncf-io\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.561987    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-daemon-config\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562006    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-lib-modules\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.564397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-log-socket\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562083    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-run-systemd\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562085    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-sys\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562126    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-sysctl-d\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562139    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-host-run-netns\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562146    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/96f8ca15-1313-46e5-bcfb-9639b2736b7b-ovnkube-config\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562154    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/baaf76a2-be45-4361-9e59-d750e56eba21-tmp\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562187    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-socket-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562195    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-sysconfig\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562232    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-modprobe-d\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562291    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-run-systemd\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562305    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-cni-dir\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562329    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-host-run-k8s-cni-cncf-io\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562379    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/96f8ca15-1313-46e5-bcfb-9639b2736b7b-log-socket\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562439    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46a71960-5367-47ee-9326-60be4aeea347-socket-dir\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562472    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/baaf76a2-be45-4361-9e59-d750e56eba21-etc-sysctl-d\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.562484    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/96f8ca15-1313-46e5-bcfb-9639b2736b7b-ovnkube-script-lib\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.565066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.564069    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-multus-daemon-config\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.566144 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.566112    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/baaf76a2-be45-4361-9e59-d750e56eba21-etc-tuned\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.567085 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.566664    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/96f8ca15-1313-46e5-bcfb-9639b2736b7b-ovn-node-metrics-cert\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.567085 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.566997    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd636fcf-c98a-4d32-a54f-10aa4648580b-agent-certs\") pod \"konnectivity-agent-cctlc\" (UID: \"bd636fcf-c98a-4d32-a54f-10aa4648580b\") " pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:18.567345 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.567178    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/baaf76a2-be45-4361-9e59-d750e56eba21-tmp\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.570025 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.569404    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn25z\" (UniqueName: \"kubernetes.io/projected/cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6-kube-api-access-mn25z\") pod \"node-ca-277dr\" (UID: \"cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6\") " pod="openshift-image-registry/node-ca-277dr"
Sep 11 19:18:18.571934 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.571905    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqzt7\" (UniqueName: \"kubernetes.io/projected/7dac3817-ba81-4281-a22f-44ebd2700b4c-kube-api-access-cqzt7\") pod \"iptables-alerter-76dkk\" (UID: \"7dac3817-ba81-4281-a22f-44ebd2700b4c\") " pod="openshift-network-operator/iptables-alerter-76dkk"
Sep 11 19:18:18.573019 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.572994    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw8jb\" (UniqueName: \"kubernetes.io/projected/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-kube-api-access-sw8jb\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:18.573183 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.573150    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkpm7\" (UniqueName: \"kubernetes.io/projected/46a71960-5367-47ee-9326-60be4aeea347-kube-api-access-vkpm7\") pod \"aws-ebs-csi-driver-node-pggl9\" (UID: \"46a71960-5367-47ee-9326-60be4aeea347\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.573429 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.573393    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nsdf\" (UniqueName: \"kubernetes.io/projected/cc814053-fb3d-45bb-bc8e-ea849d8f1951-kube-api-access-7nsdf\") pod \"multus-additional-cni-plugins-xxfp8\" (UID: \"cc814053-fb3d-45bb-bc8e-ea849d8f1951\") " pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.573994 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.573949    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m84lq\" (UniqueName: \"kubernetes.io/projected/96f8ca15-1313-46e5-bcfb-9639b2736b7b-kube-api-access-m84lq\") pod \"ovnkube-node-kfhvg\" (UID: \"96f8ca15-1313-46e5-bcfb-9639b2736b7b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.574126 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.574098    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fsxl\" (UniqueName: \"kubernetes.io/projected/3754f687-4a22-4dbe-9151-d33ccbc0afd5-kube-api-access-6fsxl\") pod \"node-resolver-7hsvz\" (UID: \"3754f687-4a22-4dbe-9151-d33ccbc0afd5\") " pod="openshift-dns/node-resolver-7hsvz"
Sep 11 19:18:18.575754 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.575735    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk6h5\" (UniqueName: \"kubernetes.io/projected/dac09145-2ebe-48a5-b5d2-c4ab764b8b9f-kube-api-access-qk6h5\") pod \"multus-km429\" (UID: \"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f\") " pod="openshift-multus/multus-km429"
Sep 11 19:18:18.575848 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.575778    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt5vj\" (UniqueName: \"kubernetes.io/projected/baaf76a2-be45-4361-9e59-d750e56eba21-kube-api-access-vt5vj\") pod \"tuned-qhppv\" (UID: \"baaf76a2-be45-4361-9e59-d750e56eba21\") " pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.663356 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.663311    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5kl\" (UniqueName: \"kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl\") pod \"network-check-target-m6tdc\" (UID: \"c9378a65-f0bc-4a82-863c-a1647b854dd7\") " pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:18.668654 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:18.668627    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 19:18:18.668654 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:18.668659    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 19:18:18.668818 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:18.668673    2583 projected.go:196] Error preparing data for projected volume kube-api-access-6s5kl for pod openshift-network-diagnostics/network-check-target-m6tdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:18.668818 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:18.668760    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl podName:c9378a65-f0bc-4a82-863c-a1647b854dd7 nodeName:}" failed. No retries permitted until 2026-09-11 19:18:19.168739854 +0000 UTC m=+3.097364745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6s5kl" (UniqueName: "kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl") pod "network-check-target-m6tdc" (UID: "c9378a65-f0bc-4a82-863c-a1647b854dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:18.746271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.746173    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qhppv"
Sep 11 19:18:18.754125 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.754102    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9"
Sep 11 19:18:18.761795 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.761773    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-277dr"
Sep 11 19:18:18.767512 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.767480    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-km429"
Sep 11 19:18:18.774117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.774097    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-76dkk"
Sep 11 19:18:18.781559 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.781534    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7hsvz"
Sep 11 19:18:18.788164 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.788142    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xxfp8"
Sep 11 19:18:18.794124 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.794107    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:18.799198 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:18.799177    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:19.066578 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.066486    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:19.066578 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.066568    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:19.066780 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:19.066678    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:19.066780 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:19.066680    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:19.066780 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:19.066736    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs podName:c1b7247e-b7a4-44db-a5dd-4817bb0df58a nodeName:}" failed. No retries permitted until 2026-09-11 19:18:20.066718546 +0000 UTC m=+3.995343422 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs") pod "network-metrics-daemon-dvg7l" (UID: "c1b7247e-b7a4-44db-a5dd-4817bb0df58a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:19.066780 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:19.066753    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret podName:c2c8ebda-08bc-4433-9b6d-e4344b72ee3b nodeName:}" failed. No retries permitted until 2026-09-11 19:18:20.066744743 +0000 UTC m=+3.995369619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret") pod "global-pull-secret-syncer-69wfq" (UID: "c2c8ebda-08bc-4433-9b6d-e4344b72ee3b") : object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:19.267852 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.267812    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5kl\" (UniqueName: \"kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl\") pod \"network-check-target-m6tdc\" (UID: \"c9378a65-f0bc-4a82-863c-a1647b854dd7\") " pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:19.268038 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:19.267986    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 19:18:19.268038 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:19.268003    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 19:18:19.268038 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:19.268015    2583 projected.go:196] Error preparing data for projected volume kube-api-access-6s5kl for pod openshift-network-diagnostics/network-check-target-m6tdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:19.268201 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:19.268072    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl podName:c9378a65-f0bc-4a82-863c-a1647b854dd7 nodeName:}" failed. No retries permitted until 2026-09-11 19:18:20.268059376 +0000 UTC m=+4.196684252 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6s5kl" (UniqueName: "kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl") pod "network-check-target-m6tdc" (UID: "c9378a65-f0bc-4a82-863c-a1647b854dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:19.333919 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:19.333893    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96f8ca15_1313_46e5_bcfb_9639b2736b7b.slice/crio-5e10fdab548214df886fce24036ca36e4edbe5fe8f878ea5ca0ae7a925be3be4 WatchSource:0}: Error finding container 5e10fdab548214df886fce24036ca36e4edbe5fe8f878ea5ca0ae7a925be3be4: Status 404 returned error can't find the container with id 5e10fdab548214df886fce24036ca36e4edbe5fe8f878ea5ca0ae7a925be3be4
Sep 11 19:18:19.357528 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:19.357501    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46a71960_5367_47ee_9326_60be4aeea347.slice/crio-009a00015be29005e4086c9d5868838ba0026afc45d21c704697a43eb82b1b8a WatchSource:0}: Error finding container 009a00015be29005e4086c9d5868838ba0026afc45d21c704697a43eb82b1b8a: Status 404 returned error can't find the container with id 009a00015be29005e4086c9d5868838ba0026afc45d21c704697a43eb82b1b8a
Sep 11 19:18:19.358609 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:19.358433    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3754f687_4a22_4dbe_9151_d33ccbc0afd5.slice/crio-ca26713a3e92fbaf187f0f8d2202657627d989e65d5de7f38662673b36d4ec84 WatchSource:0}: Error finding container ca26713a3e92fbaf187f0f8d2202657627d989e65d5de7f38662673b36d4ec84: Status 404 returned error can't find the container with id ca26713a3e92fbaf187f0f8d2202657627d989e65d5de7f38662673b36d4ec84
Sep 11 19:18:19.359883 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:19.359821    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dac3817_ba81_4281_a22f_44ebd2700b4c.slice/crio-453e2478c749a150e055e1a9257feba4b03122633e84e49e0d4b63f8c3c4adc4 WatchSource:0}: Error finding container 453e2478c749a150e055e1a9257feba4b03122633e84e49e0d4b63f8c3c4adc4: Status 404 returned error can't find the container with id 453e2478c749a150e055e1a9257feba4b03122633e84e49e0d4b63f8c3c4adc4
Sep 11 19:18:19.361560 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:19.360960    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaaf76a2_be45_4361_9e59_d750e56eba21.slice/crio-cb84030ed5e95afc4316ecfecdc09cf0261a1307981c81281f6bfac7da9c342d WatchSource:0}: Error finding container cb84030ed5e95afc4316ecfecdc09cf0261a1307981c81281f6bfac7da9c342d: Status 404 returned error can't find the container with id cb84030ed5e95afc4316ecfecdc09cf0261a1307981c81281f6bfac7da9c342d
Sep 11 19:18:19.362739 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:19.362467    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd7f61a5_358b_44ea_aeeb_8f7b8ca446f6.slice/crio-6d5d204967cf09d13c3f8d00b762117e630358bf1a077c08d28607cd3a6fa92c WatchSource:0}: Error finding container 6d5d204967cf09d13c3f8d00b762117e630358bf1a077c08d28607cd3a6fa92c: Status 404 returned error can't find the container with id 6d5d204967cf09d13c3f8d00b762117e630358bf1a077c08d28607cd3a6fa92c
Sep 11 19:18:19.362739 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:18:19.362620    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddac09145_2ebe_48a5_b5d2_c4ab764b8b9f.slice/crio-bd08ed56be6ce2c81a151902f9e59c019faa2eb884e86327210408e80a9a6094 WatchSource:0}: Error finding container bd08ed56be6ce2c81a151902f9e59c019faa2eb884e86327210408e80a9a6094: Status 404 returned error can't find the container with id bd08ed56be6ce2c81a151902f9e59c019faa2eb884e86327210408e80a9a6094
Sep 11 19:18:19.487415 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.487372    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 19:13:18 +0000 UTC" deadline="2028-06-14 14:07:11.028449456 +0000 UTC"
Sep 11 19:18:19.487415 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.487411    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15402h48m51.541043522s"
Sep 11 19:18:19.548040 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.548013    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7hsvz" event={"ID":"3754f687-4a22-4dbe-9151-d33ccbc0afd5","Type":"ContainerStarted","Data":"ca26713a3e92fbaf187f0f8d2202657627d989e65d5de7f38662673b36d4ec84"}
Sep 11 19:18:19.551150 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.551074    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-277dr" event={"ID":"cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6","Type":"ContainerStarted","Data":"6d5d204967cf09d13c3f8d00b762117e630358bf1a077c08d28607cd3a6fa92c"}
Sep 11 19:18:19.552217 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.552187    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cctlc" event={"ID":"bd636fcf-c98a-4d32-a54f-10aa4648580b","Type":"ContainerStarted","Data":"61e75a0568ac3515ea19ec44557c466abe5a438cd6e00b87a9214e3f704b7256"}
Sep 11 19:18:19.553459 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.553434    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxfp8" event={"ID":"cc814053-fb3d-45bb-bc8e-ea849d8f1951","Type":"ContainerStarted","Data":"cde1a8de3060a4c44a60b258efd7e1ecd1ab010f278cb6edb7090c86d302c05b"}
Sep 11 19:18:19.554556 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.554527    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg" event={"ID":"96f8ca15-1313-46e5-bcfb-9639b2736b7b","Type":"ContainerStarted","Data":"5e10fdab548214df886fce24036ca36e4edbe5fe8f878ea5ca0ae7a925be3be4"}
Sep 11 19:18:19.556660 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.556635    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-147.ec2.internal" event={"ID":"64cf4cd8df0a6677bea382d1c5122f9a","Type":"ContainerStarted","Data":"bbcc669201fe7bafea681fec05c129170342d1bbcc6af39d0e752d9919ca7ba3"}
Sep 11 19:18:19.557761 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.557738    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qhppv" event={"ID":"baaf76a2-be45-4361-9e59-d750e56eba21","Type":"ContainerStarted","Data":"cb84030ed5e95afc4316ecfecdc09cf0261a1307981c81281f6bfac7da9c342d"}
Sep 11 19:18:19.558760 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.558740    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-76dkk" event={"ID":"7dac3817-ba81-4281-a22f-44ebd2700b4c","Type":"ContainerStarted","Data":"453e2478c749a150e055e1a9257feba4b03122633e84e49e0d4b63f8c3c4adc4"}
Sep 11 19:18:19.559737 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.559719    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9" event={"ID":"46a71960-5367-47ee-9326-60be4aeea347","Type":"ContainerStarted","Data":"009a00015be29005e4086c9d5868838ba0026afc45d21c704697a43eb82b1b8a"}
Sep 11 19:18:19.560758 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.560741    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-km429" event={"ID":"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f","Type":"ContainerStarted","Data":"bd08ed56be6ce2c81a151902f9e59c019faa2eb884e86327210408e80a9a6094"}
Sep 11 19:18:19.569768 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:19.569695    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-147.ec2.internal" podStartSLOduration=1.56967893 podStartE2EDuration="1.56967893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:18:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:18:19.56945304 +0000 UTC m=+3.498077931" watchObservedRunningTime="2026-09-11 19:18:19.56967893 +0000 UTC m=+3.498303818"
Sep 11 19:18:20.076126 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:20.075392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:20.076126 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:20.075695    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:20.076126 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:20.075840    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:20.076126 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:20.075902    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs podName:c1b7247e-b7a4-44db-a5dd-4817bb0df58a nodeName:}" failed. No retries permitted until 2026-09-11 19:18:22.075883653 +0000 UTC m=+6.004508549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs") pod "network-metrics-daemon-dvg7l" (UID: "c1b7247e-b7a4-44db-a5dd-4817bb0df58a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:20.076495 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:20.076344    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:20.076495 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:20.076401    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret podName:c2c8ebda-08bc-4433-9b6d-e4344b72ee3b nodeName:}" failed. No retries permitted until 2026-09-11 19:18:22.076384809 +0000 UTC m=+6.005009699 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret") pod "global-pull-secret-syncer-69wfq" (UID: "c2c8ebda-08bc-4433-9b6d-e4344b72ee3b") : object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:20.277858 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:20.277820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5kl\" (UniqueName: \"kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl\") pod \"network-check-target-m6tdc\" (UID: \"c9378a65-f0bc-4a82-863c-a1647b854dd7\") " pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:20.288747 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:20.288703    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 19:18:20.288747 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:20.288753    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 19:18:20.288959 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:20.288769    2583 projected.go:196] Error preparing data for projected volume kube-api-access-6s5kl for pod openshift-network-diagnostics/network-check-target-m6tdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:20.288959 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:20.288867    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl podName:c9378a65-f0bc-4a82-863c-a1647b854dd7 nodeName:}" failed. No retries permitted until 2026-09-11 19:18:22.288844143 +0000 UTC m=+6.217469037 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6s5kl" (UniqueName: "kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl") pod "network-check-target-m6tdc" (UID: "c9378a65-f0bc-4a82-863c-a1647b854dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:20.488276 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:20.488146    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 19:13:18 +0000 UTC" deadline="2028-04-03 21:08:52.140822011 +0000 UTC"
Sep 11 19:18:20.488276 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:20.488214    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13681h50m31.652611961s"
Sep 11 19:18:20.543731 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:20.543701    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:20.543886 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:20.543839    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:20.544278 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:20.544258    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:20.544375 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:20.544352    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:20.544428 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:20.544415    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:20.544502 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:20.544482    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:20.586156 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:20.586065    2583 generic.go:356] "Generic (PLEG): container finished" podID="6416ebedd89de9f7c7c06d3da5392139" containerID="21de6a1683136682a3c495ed0f970b4bd4f2a869ebbb40192d792a77e6935d55" exitCode=0
Sep 11 19:18:20.586156 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:20.586142    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal" event={"ID":"6416ebedd89de9f7c7c06d3da5392139","Type":"ContainerDied","Data":"21de6a1683136682a3c495ed0f970b4bd4f2a869ebbb40192d792a77e6935d55"}
Sep 11 19:18:21.634727 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:21.634691    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal" event={"ID":"6416ebedd89de9f7c7c06d3da5392139","Type":"ContainerStarted","Data":"2c0e87bbcaf22623148e25d1ea1163dfea401a01b184c26750f85dd00df39f62"}
Sep 11 19:18:22.095266 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:22.094495    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:22.095266 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:22.094571    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:22.095266 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:22.094709    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:22.095266 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:22.094765    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret podName:c2c8ebda-08bc-4433-9b6d-e4344b72ee3b nodeName:}" failed. No retries permitted until 2026-09-11 19:18:26.094747572 +0000 UTC m=+10.023372454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret") pod "global-pull-secret-syncer-69wfq" (UID: "c2c8ebda-08bc-4433-9b6d-e4344b72ee3b") : object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:22.095266 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:22.095129    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:22.095266 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:22.095174    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs podName:c1b7247e-b7a4-44db-a5dd-4817bb0df58a nodeName:}" failed. No retries permitted until 2026-09-11 19:18:26.095159319 +0000 UTC m=+10.023784209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs") pod "network-metrics-daemon-dvg7l" (UID: "c1b7247e-b7a4-44db-a5dd-4817bb0df58a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:22.296701 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:22.295967    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5kl\" (UniqueName: \"kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl\") pod \"network-check-target-m6tdc\" (UID: \"c9378a65-f0bc-4a82-863c-a1647b854dd7\") " pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:22.296701 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:22.296125    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 19:18:22.296701 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:22.296144    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 19:18:22.296701 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:22.296155    2583 projected.go:196] Error preparing data for projected volume kube-api-access-6s5kl for pod openshift-network-diagnostics/network-check-target-m6tdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:22.296701 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:22.296219    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl podName:c9378a65-f0bc-4a82-863c-a1647b854dd7 nodeName:}" failed. No retries permitted until 2026-09-11 19:18:26.296197636 +0000 UTC m=+10.224822520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6s5kl" (UniqueName: "kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl") pod "network-check-target-m6tdc" (UID: "c9378a65-f0bc-4a82-863c-a1647b854dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:22.540430 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:22.540344    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:22.540583 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:22.540484    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:22.540583 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:22.540540    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:22.540679 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:22.540626    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:22.540800 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:22.540766    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:22.540882 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:22.540844    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:24.540965 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:24.540923    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:24.541347 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:24.541068    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:24.541561 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:24.541543    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:24.541658 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:24.541638    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:24.541754 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:24.541741    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:24.541842 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:24.541825    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:26.137517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:26.137219    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:26.137517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:26.137320    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:26.137517 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:26.137374    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:26.137517 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:26.137448    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:26.137517 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:26.137468    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret podName:c2c8ebda-08bc-4433-9b6d-e4344b72ee3b nodeName:}" failed. No retries permitted until 2026-09-11 19:18:34.137438153 +0000 UTC m=+18.066063052 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret") pod "global-pull-secret-syncer-69wfq" (UID: "c2c8ebda-08bc-4433-9b6d-e4344b72ee3b") : object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:26.137517 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:26.137494    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs podName:c1b7247e-b7a4-44db-a5dd-4817bb0df58a nodeName:}" failed. No retries permitted until 2026-09-11 19:18:34.137482564 +0000 UTC m=+18.066107439 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs") pod "network-metrics-daemon-dvg7l" (UID: "c1b7247e-b7a4-44db-a5dd-4817bb0df58a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:26.339450 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:26.338891    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5kl\" (UniqueName: \"kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl\") pod \"network-check-target-m6tdc\" (UID: \"c9378a65-f0bc-4a82-863c-a1647b854dd7\") " pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:26.339450 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:26.339048    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 19:18:26.339450 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:26.339075    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 19:18:26.339450 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:26.339088    2583 projected.go:196] Error preparing data for projected volume kube-api-access-6s5kl for pod openshift-network-diagnostics/network-check-target-m6tdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:26.339450 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:26.339150    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl podName:c9378a65-f0bc-4a82-863c-a1647b854dd7 nodeName:}" failed. No retries permitted until 2026-09-11 19:18:34.339131666 +0000 UTC m=+18.267756551 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6s5kl" (UniqueName: "kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl") pod "network-check-target-m6tdc" (UID: "c9378a65-f0bc-4a82-863c-a1647b854dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:26.541475 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:26.541406    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:26.541614 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:26.541516    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:26.541614 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:26.541582    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:26.541683 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:26.541660    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:26.541719 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:26.541677    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:26.541803 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:26.541785    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:28.540069 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:28.540031    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:28.540595 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:28.540040    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:28.540595 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:28.540177    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:28.540595 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:28.540196    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:28.540595 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:28.540305    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:28.540595 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:28.540389    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:30.540673 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:30.540632    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:30.540673 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:30.540666    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:30.541155 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:30.540640    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:30.541155 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:30.540768    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:30.541155 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:30.540846    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:30.541155 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:30.540914    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:32.540337 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:32.540297    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:32.540785 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:32.540297    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:32.540785 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:32.540440    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:32.540785 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:32.540302    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:32.540785 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:32.540511    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:32.540785 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:32.540561    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:34.201008 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:34.200953    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:34.201518 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:34.201028    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:34.201518 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:34.201161    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:34.201518 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:34.201215    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs podName:c1b7247e-b7a4-44db-a5dd-4817bb0df58a nodeName:}" failed. No retries permitted until 2026-09-11 19:18:50.201200002 +0000 UTC m=+34.129824878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs") pod "network-metrics-daemon-dvg7l" (UID: "c1b7247e-b7a4-44db-a5dd-4817bb0df58a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:34.201705 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:34.201621    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:34.201705 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:34.201680    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret podName:c2c8ebda-08bc-4433-9b6d-e4344b72ee3b nodeName:}" failed. No retries permitted until 2026-09-11 19:18:50.201663879 +0000 UTC m=+34.130288772 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret") pod "global-pull-secret-syncer-69wfq" (UID: "c2c8ebda-08bc-4433-9b6d-e4344b72ee3b") : object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:34.402722 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:34.402675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5kl\" (UniqueName: \"kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl\") pod \"network-check-target-m6tdc\" (UID: \"c9378a65-f0bc-4a82-863c-a1647b854dd7\") " pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:34.402899 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:34.402850    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 19:18:34.402899 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:34.402880    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 19:18:34.402899 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:34.402895    2583 projected.go:196] Error preparing data for projected volume kube-api-access-6s5kl for pod openshift-network-diagnostics/network-check-target-m6tdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:34.403029 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:34.402970    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl podName:c9378a65-f0bc-4a82-863c-a1647b854dd7 nodeName:}" failed. No retries permitted until 2026-09-11 19:18:50.402950224 +0000 UTC m=+34.331575131 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6s5kl" (UniqueName: "kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl") pod "network-check-target-m6tdc" (UID: "c9378a65-f0bc-4a82-863c-a1647b854dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:34.540101 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:34.540017    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:34.540101 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:34.540025    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:34.540101 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:34.540068    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:34.540403 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:34.540177    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:34.540403 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:34.540318    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:34.540403 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:34.540397    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:36.541135 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:36.541104    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:36.541671 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:36.541207    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:36.541671 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:36.541215    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:36.541671 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:36.541269    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:36.541671 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:36.541350    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:36.541671 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:36.541446    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:37.675619 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.672039    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qhppv" event={"ID":"baaf76a2-be45-4361-9e59-d750e56eba21","Type":"ContainerStarted","Data":"36f02a0360274d8674f23a29709817ab9e3668251bb11b7459435ff9eb54f236"}
Sep 11 19:18:37.688889 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.688816    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-km429" event={"ID":"dac09145-2ebe-48a5-b5d2-c4ab764b8b9f","Type":"ContainerStarted","Data":"5b56f8b2ca92a79cf8b226000de99b01889b54651eee775c6657a4d4a74ef97e"}
Sep 11 19:18:37.691464 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.691432    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7hsvz" event={"ID":"3754f687-4a22-4dbe-9151-d33ccbc0afd5","Type":"ContainerStarted","Data":"dd048189609dc757ae5d9e8beccee399a9b1193b014c52f2dc6cb21d3447930f"}
Sep 11 19:18:37.693257 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.693192    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-147.ec2.internal" podStartSLOduration=19.693176166 podStartE2EDuration="19.693176166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:18:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:18:21.648956397 +0000 UTC m=+5.577581325" watchObservedRunningTime="2026-09-11 19:18:37.693176166 +0000 UTC m=+21.621801064"
Sep 11 19:18:37.695997 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.695972    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cctlc" event={"ID":"bd636fcf-c98a-4d32-a54f-10aa4648580b","Type":"ContainerStarted","Data":"19afc6b1de9bf3970cc186ab0fbda618f3168f22e7796e343c2c54efba2fe4c5"}
Sep 11 19:18:37.698120 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.698087    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxfp8" event={"ID":"cc814053-fb3d-45bb-bc8e-ea849d8f1951","Type":"ContainerStarted","Data":"af0555e8abcf3efd46c1fcc1d3395209056e531fd010758d75b959ca0c3d401e"}
Sep 11 19:18:37.700635 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.700609    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg" event={"ID":"96f8ca15-1313-46e5-bcfb-9639b2736b7b","Type":"ContainerStarted","Data":"760b6b88b5364759a565073af0b8ac31a89bd3b02309c940283889f1612fe1b8"}
Sep 11 19:18:37.712426 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.712230    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-km429" podStartSLOduration=3.574760983 podStartE2EDuration="21.712215212s" totalImagesPullingTime="18.137454229s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:18:37.712060233 +0000 UTC m=+21.640685142" watchObservedRunningTime="2026-09-11 19:18:37.712215212 +0000 UTC m=+21.640840109"
Sep 11 19:18:37.712767 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.712735    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qhppv" podStartSLOduration=3.636944464 podStartE2EDuration="21.712726004s" totalImagesPullingTime="18.07578154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:18:37.693309348 +0000 UTC m=+21.621934247" watchObservedRunningTime="2026-09-11 19:18:37.712726004 +0000 UTC m=+21.641350904"
Sep 11 19:18:37.727458 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.727412    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7hsvz" podStartSLOduration=3.66378265 podStartE2EDuration="21.727397278s" totalImagesPullingTime="18.063614628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:18:37.727221435 +0000 UTC m=+21.655846335" watchObservedRunningTime="2026-09-11 19:18:37.727397278 +0000 UTC m=+21.656022175"
Sep 11 19:18:37.742047 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.741995    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cctlc" podStartSLOduration=7.583866887 podStartE2EDuration="21.74198062s" totalImagesPullingTime="14.158113733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:18:37.741547815 +0000 UTC m=+21.670172713" watchObservedRunningTime="2026-09-11 19:18:37.74198062 +0000 UTC m=+21.670605518"
Sep 11 19:18:37.755823 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.755777    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-277dr" podStartSLOduration=3.681802088 podStartE2EDuration="21.755762061s" totalImagesPullingTime="18.073959973s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:18:37.755605988 +0000 UTC m=+21.684230887" watchObservedRunningTime="2026-09-11 19:18:37.755762061 +0000 UTC m=+21.684386959"
Sep 11 19:18:37.785453 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.785420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:37.848794 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.848703    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:37.949488 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:37.949459    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:38.540512 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.540476    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:38.540737 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.540479    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:38.540737 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:38.540604    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:38.540737 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:38.540695    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:38.540737 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.540484    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:38.540949 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:38.540772    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:38.704497 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.704432    2583 generic.go:356] "Generic (PLEG): container finished" podID="cc814053-fb3d-45bb-bc8e-ea849d8f1951" containerID="af0555e8abcf3efd46c1fcc1d3395209056e531fd010758d75b959ca0c3d401e" exitCode=0
Sep 11 19:18:38.705017 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.704511    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxfp8" event={"ID":"cc814053-fb3d-45bb-bc8e-ea849d8f1951","Type":"ContainerDied","Data":"af0555e8abcf3efd46c1fcc1d3395209056e531fd010758d75b959ca0c3d401e"}
Sep 11 19:18:38.707712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.707685    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg" event={"ID":"96f8ca15-1313-46e5-bcfb-9639b2736b7b","Type":"ContainerStarted","Data":"84c9d6736040d846eda5962b7a9071e1df9984dcb5bb995c0de7c807f63d6643"}
Sep 11 19:18:38.707791 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.707764    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg" event={"ID":"96f8ca15-1313-46e5-bcfb-9639b2736b7b","Type":"ContainerStarted","Data":"ff31c1dc61136ef93307cf73401d85e9e49b8626fd7b4226666a3c26255b1b3e"}
Sep 11 19:18:38.707834 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.707805    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg" event={"ID":"96f8ca15-1313-46e5-bcfb-9639b2736b7b","Type":"ContainerStarted","Data":"b15e89ae4822a5ff2ea5fdc61d3c52562ce5202c5a6f41d632be096ab77aa3f8"}
Sep 11 19:18:38.707868 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.707834    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg" event={"ID":"96f8ca15-1313-46e5-bcfb-9639b2736b7b","Type":"ContainerStarted","Data":"9f0d63ec3e8d6177cc944f738ad8d5dc5d358d10ac26ec85f727b42ca6fcdadb"}
Sep 11 19:18:38.707900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.707869    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg" event={"ID":"96f8ca15-1313-46e5-bcfb-9639b2736b7b","Type":"ContainerStarted","Data":"6e60b1703401c496107f5e9703b25f8f4259f895fc34133a8e9208984955d320"}
Sep 11 19:18:38.709255 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.709186    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-76dkk" event={"ID":"7dac3817-ba81-4281-a22f-44ebd2700b4c","Type":"ContainerStarted","Data":"cb1d8fab355d8f308ded790a0d470828676a33c63112c1cf8fdb428435fef06a"}
Sep 11 19:18:38.710695 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.710665    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9" event={"ID":"46a71960-5367-47ee-9326-60be4aeea347","Type":"ContainerStarted","Data":"7f59df19ce3036a0cc16eb08e27f6566a026f01b79f226f5ead3cd0123a3547d"}
Sep 11 19:18:38.711936 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.711916    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-277dr" event={"ID":"cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6","Type":"ContainerStarted","Data":"ff4acd32d103525e890f2a57867f2e105fbce10a983d047f857b8ee56ae75055"}
Sep 11 19:18:38.736764 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.736717    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-76dkk" podStartSLOduration=4.674192965 podStartE2EDuration="22.736703871s" totalImagesPullingTime="18.062510906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:18:38.736352934 +0000 UTC m=+22.664977832" watchObservedRunningTime="2026-09-11 19:18:38.736703871 +0000 UTC m=+22.665328769"
Sep 11 19:18:38.858389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.858357    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:38.960543 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:38.960481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:39.092834 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:39.092804    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 19:18:39.524379 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:39.524292    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T19:18:39.092827802Z","UUID":"9e159301-d363-444f-acfa-86aec5646155","Handler":null,"Name":"","Endpoint":""}
Sep 11 19:18:39.525750 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:39.525728    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 19:18:39.525864 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:39.525761    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 19:18:39.716824 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:39.716782    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9" event={"ID":"46a71960-5367-47ee-9326-60be4aeea347","Type":"ContainerStarted","Data":"5f2d0b31535e37a2125b191c4abda3a3b42ce57283e3e9adef5a24ddc38b8616"}
Sep 11 19:18:40.540420 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:40.540223    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:40.540420 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:40.540359    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:40.540420 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:40.540369    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:40.540751 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:40.540730    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:40.540835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:40.540761    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:40.540879 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:40.540864    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:40.722034 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:40.721993    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg" event={"ID":"96f8ca15-1313-46e5-bcfb-9639b2736b7b","Type":"ContainerStarted","Data":"f44511ec5a1e014848ab51f97ab47b1e5ef01b572577bbcbab6c623003b7b974"}
Sep 11 19:18:40.723926 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:40.723903    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9" event={"ID":"46a71960-5367-47ee-9326-60be4aeea347","Type":"ContainerStarted","Data":"0d69fca51fa74036c64a1901ccf55f60ffa2ef4f86653a175667ea0b5b6b60c4"}
Sep 11 19:18:40.741651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:40.741602    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pggl9" podStartSLOduration=3.979738285 podStartE2EDuration="24.741588774s" totalImagesPullingTime="20.761850489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:18:40.741068132 +0000 UTC m=+24.669693029" watchObservedRunningTime="2026-09-11 19:18:40.741588774 +0000 UTC m=+24.670213675"
Sep 11 19:18:41.719067 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:41.718868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:41.780616 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:41.780568    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:42.502760 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:42.502723    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:42.503486 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:42.503448    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:42.540330 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:42.540289    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:42.540330 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:42.540310    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:42.540560 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:42.540412    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:42.540560 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:42.540493    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:42.540560 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:42.540538    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:42.540680 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:42.540620    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:42.725459 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:42.725380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:42.729188 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:42.729163    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:42.729722 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:42.729702    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cctlc"
Sep 11 19:18:42.825534 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:42.825486    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:42.894582 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:42.894546    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:43.652835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:43.652799    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:43.727089 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:43.727058    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:43.733319 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:43.733291    2583 generic.go:356] "Generic (PLEG): container finished" podID="cc814053-fb3d-45bb-bc8e-ea849d8f1951" containerID="9d6e5f299e9e25e4830850ab271019a025903b24cc12b0412da88a0cdbcbc34d" exitCode=0
Sep 11 19:18:43.733435 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:43.733331    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxfp8" event={"ID":"cc814053-fb3d-45bb-bc8e-ea849d8f1951","Type":"ContainerDied","Data":"9d6e5f299e9e25e4830850ab271019a025903b24cc12b0412da88a0cdbcbc34d"}
Sep 11 19:18:43.737464 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:43.737442    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg" event={"ID":"96f8ca15-1313-46e5-bcfb-9639b2736b7b","Type":"ContainerStarted","Data":"38ef722330b2c64f120f4d3e033c7ea90a2b55ae6360cfe0001a4d7fa9269f19"}
Sep 11 19:18:43.738233 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:43.737830    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:43.738318 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:43.738264    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:43.739964 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:43.739938    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:43.755099 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:43.755040    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:43.755580 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:43.755563    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:18:43.782866 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:43.782826    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg" podStartSLOduration=9.639345565 podStartE2EDuration="27.782814583s" totalImagesPullingTime="18.143469018s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:18:43.782670011 +0000 UTC m=+27.711294910" watchObservedRunningTime="2026-09-11 19:18:43.782814583 +0000 UTC m=+27.711439480"
Sep 11 19:18:44.252862 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:44.252612    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:44.506955 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:44.506875    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:44.540524 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:44.540485    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:44.540707 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:44.540485    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:44.540707 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:44.540640    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:44.540707 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:44.540684    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:44.540707 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:44.540489    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:44.540898 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:44.540761    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:44.593447 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:44.593360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:44.656498 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:44.656472    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:44.757007 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:44.756945    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:44.801310 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:44.801290    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7hsvz_3754f687-4a22-4dbe-9151-d33ccbc0afd5/dns-node-resolver/0.log"
Sep 11 19:18:44.964462 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:44.964361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:45.062360 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.062275    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:45.219983 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.219950    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m6tdc"]
Sep 11 19:18:45.220116 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.220043    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:45.220150 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:45.220123    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:45.223169 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.223126    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-69wfq"]
Sep 11 19:18:45.223350 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.223235    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:45.223416 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:45.223370    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:45.223825 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.223797    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dvg7l"]
Sep 11 19:18:45.223966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.223881    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:45.223966 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:45.223957    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:45.517155 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.517125    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:45.571321 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.571278    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:45.601689 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.601665    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-277dr_cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6/node-ca/0.log"
Sep 11 19:18:45.744685 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.744649    2583 generic.go:356] "Generic (PLEG): container finished" podID="cc814053-fb3d-45bb-bc8e-ea849d8f1951" containerID="4da2fe2895a1495ebf0b88bf1e8a756622fcc3c9c83ccbc1bac60d4e149700c7" exitCode=0
Sep 11 19:18:45.744828 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:45.744735    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxfp8" event={"ID":"cc814053-fb3d-45bb-bc8e-ea849d8f1951","Type":"ContainerDied","Data":"4da2fe2895a1495ebf0b88bf1e8a756622fcc3c9c83ccbc1bac60d4e149700c7"}
Sep 11 19:18:46.540635 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:46.540602    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:46.541005 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:46.540676    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:46.541005 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:46.540701    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:46.541005 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:46.540832    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:46.674206 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:46.674171    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:46.749753 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:46.749716    2583 generic.go:356] "Generic (PLEG): container finished" podID="cc814053-fb3d-45bb-bc8e-ea849d8f1951" containerID="40430a1526415e23a280d6b9a0684d8a8fa568333d3290364b1c87cb804453e6" exitCode=0
Sep 11 19:18:46.749893 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:46.749780    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxfp8" event={"ID":"cc814053-fb3d-45bb-bc8e-ea849d8f1951","Type":"ContainerDied","Data":"40430a1526415e23a280d6b9a0684d8a8fa568333d3290364b1c87cb804453e6"}
Sep 11 19:18:47.540587 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:47.540386    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:47.540723 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:47.540690    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:47.604307 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:47.604267    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:47.778445 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:47.778395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:48.293902 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:48.293822    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:48.540752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:48.540718    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:48.541209 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:48.540836    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:48.541209 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:48.540933    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:48.541209 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:48.541041    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:48.565400 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:48.565363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:48.662701 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:48.662671    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:48.735450 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:48.735418    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:48.899845 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:48.899808    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:49.000619 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:49.000581    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:49.540170 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:49.540138    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:49.540334 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:49.540263    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:49.785195 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:49.785158    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:50.220228 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:50.220188    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:50.220465 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:50.220288    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:50.220465 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:50.220365    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:50.220465 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:50.220379    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:50.220465 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:50.220428    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs podName:c1b7247e-b7a4-44db-a5dd-4817bb0df58a nodeName:}" failed. No retries permitted until 2026-09-11 19:19:22.220413536 +0000 UTC m=+66.149038411 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs") pod "network-metrics-daemon-dvg7l" (UID: "c1b7247e-b7a4-44db-a5dd-4817bb0df58a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 19:18:50.220465 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:50.220440    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret podName:c2c8ebda-08bc-4433-9b6d-e4344b72ee3b nodeName:}" failed. No retries permitted until 2026-09-11 19:19:22.220434395 +0000 UTC m=+66.149059270 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret") pod "global-pull-secret-syncer-69wfq" (UID: "c2c8ebda-08bc-4433-9b6d-e4344b72ee3b") : object "kube-system"/"original-pull-secret" not registered
Sep 11 19:18:50.386723 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:50.386683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:50.421736 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:50.421702    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5kl\" (UniqueName: \"kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl\") pod \"network-check-target-m6tdc\" (UID: \"c9378a65-f0bc-4a82-863c-a1647b854dd7\") " pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:50.421916 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:50.421879    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 19:18:50.421916 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:50.421900    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 19:18:50.421916 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:50.421913    2583 projected.go:196] Error preparing data for projected volume kube-api-access-6s5kl for pod openshift-network-diagnostics/network-check-target-m6tdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:50.422071 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:50.421966    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl podName:c9378a65-f0bc-4a82-863c-a1647b854dd7 nodeName:}" failed. No retries permitted until 2026-09-11 19:19:22.421951527 +0000 UTC m=+66.350576403 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6s5kl" (UniqueName: "kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl") pod "network-check-target-m6tdc" (UID: "c9378a65-f0bc-4a82-863c-a1647b854dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 19:18:50.491159 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:50.491079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:50.540501 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:50.540468    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:50.540662 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:50.540468    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:50.540662 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:50.540612    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:50.540784 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:50.540697    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:50.589008 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:50.588972    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:50.786791 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:50.786704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:50.984787 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:50.984745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:51.539957 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:51.539925    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:51.540132 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:51.540029    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:51.791415 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:51.791344    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:51.982211 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:51.982177    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:52.043331 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:52.043234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:52.177719 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:52.177693    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:52.540335 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:52.540310    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:52.540469 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:52.540312    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:52.540469 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:52.540416    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:52.540555 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:52.540503    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:52.620628 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:52.620599    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:52.721394 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:52.721363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:52.765919 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:52.765882    2583 generic.go:356] "Generic (PLEG): container finished" podID="cc814053-fb3d-45bb-bc8e-ea849d8f1951" containerID="8f19deddd10e4db33c6435db7b8c356686cc4d08a2ab9bb47072227af9a23ae5" exitCode=0
Sep 11 19:18:52.766102 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:52.765962    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxfp8" event={"ID":"cc814053-fb3d-45bb-bc8e-ea849d8f1951","Type":"ContainerDied","Data":"8f19deddd10e4db33c6435db7b8c356686cc4d08a2ab9bb47072227af9a23ae5"}
Sep 11 19:18:52.791033 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:52.791003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:52.851022 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:52.850991    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:53.130307 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:53.130206    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:53.389866 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:53.389792    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:53.540058 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:53.540022    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:53.540210 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:53.540124    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:53.771439 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:53.771362    2583 generic.go:356] "Generic (PLEG): container finished" podID="cc814053-fb3d-45bb-bc8e-ea849d8f1951" containerID="4896b5a905e8d466b8caa2ce8f856e8acbc1d723b0b61f16426c7be83786a998" exitCode=0
Sep 11 19:18:53.771439 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:53.771415    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxfp8" event={"ID":"cc814053-fb3d-45bb-bc8e-ea849d8f1951","Type":"ContainerDied","Data":"4896b5a905e8d466b8caa2ce8f856e8acbc1d723b0b61f16426c7be83786a998"}
Sep 11 19:18:53.982703 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:53.982673    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:54.174901 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:54.174869    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:54.382055 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:54.382023    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:54.482305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:54.482220    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:54.540589 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:54.540512    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:54.540730 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:54.540641    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:54.540730 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:54.540665    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:54.540816 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:54.540763    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:54.582602 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:54.582562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:54.776442 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:54.776367    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxfp8" event={"ID":"cc814053-fb3d-45bb-bc8e-ea849d8f1951","Type":"ContainerStarted","Data":"28e3a6b1f2cefb2c3d17e753874409eb48ed9a9cfeb05a6c327a202dff946cbe"}
Sep 11 19:18:54.797098 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:54.797058    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xxfp8" podStartSLOduration=12.98747771 podStartE2EDuration="38.797043076s" totalImagesPullingTime="25.809565366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:18:54.797022678 +0000 UTC m=+38.725647577" watchObservedRunningTime="2026-09-11 19:18:54.797043076 +0000 UTC m=+38.725667952"
Sep 11 19:18:55.525047 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:55.525017    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:55.540138 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:55.540111    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:55.540291 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:55.540223    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:55.778998 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:55.778902    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:55.979724 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:55.979688    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:56.174499 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:56.174471    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:56.378033 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:56.377995    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:56.540988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:56.540898    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:56.540988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:56.540960    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:56.540988 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:56.540979    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:56.541584 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:56.541036    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:56.581393 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:56.581362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:57.540805 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:57.540770    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:57.540978 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:57.540869    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:18:57.580550 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:57.580516    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:57.982744 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:57.982711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:58.182316 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:58.182287    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:58.377216 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:58.377178    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:58.540875 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:58.540848    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:18:58.541040 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:58.540955    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:18:58.541040 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:58.541027    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:18:58.541137 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:58.541117    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:18:58.789837 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:58.789750    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:58.995855 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:58.995803    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:59.338952 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:59.338922    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:59.489142 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:59.489109    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:18:59.540594 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:18:59.540565    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:18:59.540691 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:18:59.540674    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:19:00.542926 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:00.542898    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:19:00.543305 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:00.542998    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:19:00.543305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:00.543049    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:19:00.543305 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:00.543136    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:19:01.540125 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:01.540094    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:19:01.540324 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:01.540191    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:19:02.540338 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:02.540306    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:19:02.540792 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:02.540411    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:19:02.540792 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:02.540502    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:19:02.540792 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:02.540623    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:19:03.540321 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:03.540225    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:19:03.540461 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:03.540338    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:19:04.540532 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:04.540498    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:19:04.540921 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:04.540640    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:19:04.540921 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:04.540676    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:19:04.540921 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:04.540765    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:19:05.540656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:05.540624    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:19:05.541094 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:05.540737    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:19:06.540987 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:06.540960    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:19:06.541401 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:06.541066    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:19:06.541401 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:06.541138    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:19:06.541401 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:06.541228    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:19:07.540321 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:07.540285    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:19:07.540519 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:07.540401    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6tdc" podUID="c9378a65-f0bc-4a82-863c-a1647b854dd7"
Sep 11 19:19:07.680046 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:07.680011    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:07.777756 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:07.777710    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:07.877426 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:07.877395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:08.037214 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:08.037183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:08.424961 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:08.424915    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:08.540714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:08.540676    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:19:08.540903 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:08.540676    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:19:08.540903 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:08.540783    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69wfq" podUID="c2c8ebda-08bc-4433-9b6d-e4344b72ee3b"
Sep 11 19:19:08.540903 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:08.540880    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvg7l" podUID="c1b7247e-b7a4-44db-a5dd-4817bb0df58a"
Sep 11 19:19:08.667197 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:08.667152    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:08.776196 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:08.776100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:08.894071 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:08.894004    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:09.001356 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.001324    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:09.090585 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.090493    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:09.189195 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.189163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:09.391655 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.391629    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:09.394052 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.394034    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-147.ec2.internal" event="NodeReady"
Sep 11 19:19:09.394181 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.394170    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 19:19:09.454503 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.454472    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zkswt"]
Sep 11 19:19:09.456090 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.456072    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5jj8q"]
Sep 11 19:19:09.456217 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.456194    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zkswt"
Sep 11 19:19:09.458085 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.458063    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.460061 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.460044    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 19:19:09.460173 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.460044    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ndgb2\""
Sep 11 19:19:09.460173 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.460151    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 19:19:09.462370 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.462353    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 19:19:09.463302 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.463258    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 19:19:09.464041 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.463707    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 19:19:09.464041 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.463727    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pjmhv\""
Sep 11 19:19:09.464041 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.463763    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 19:19:09.464041 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.463816    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 19:19:09.471006 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.470986    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zkswt"]
Sep 11 19:19:09.473058 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.473040    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5jj8q"]
Sep 11 19:19:09.478877 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.478860    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-49ldf"]
Sep 11 19:19:09.480540 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.480524    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.486993 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.486972    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-257h7\""
Sep 11 19:19:09.487157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.487117    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 19:19:09.487300 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.487285    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 19:19:09.501210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.501186    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-49ldf"]
Sep 11 19:19:09.539969 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.539942    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:19:09.542372 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.542346    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 19:19:09.542473 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.542384    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tdnq7\""
Sep 11 19:19:09.542473 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.542393    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 19:19:09.555129 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.555108    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d2a55d8-22f4-48cd-8112-59c9328b9e05-cert\") pod \"ingress-canary-zkswt\" (UID: \"2d2a55d8-22f4-48cd-8112-59c9328b9e05\") " pod="openshift-ingress-canary/ingress-canary-zkswt"
Sep 11 19:19:09.555220 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.555134    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4162f7-09dd-44d1-85dd-d1380adfadef-config-volume\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.555220 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.555158    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxtwt\" (UniqueName: \"kubernetes.io/projected/c5eb3f3d-bd85-4887-8f0a-3173892dac83-kube-api-access-zxtwt\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.555220 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.555201    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5eb3f3d-bd85-4887-8f0a-3173892dac83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.555337 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.555264    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5eb3f3d-bd85-4887-8f0a-3173892dac83-data-volume\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.555337 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.555282    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4162f7-09dd-44d1-85dd-d1380adfadef-tmp-dir\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.555337 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.555298    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e4162f7-09dd-44d1-85dd-d1380adfadef-metrics-tls\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.555337 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.555317    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5eb3f3d-bd85-4887-8f0a-3173892dac83-crio-socket\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.555447 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.555352    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-562hw\" (UniqueName: \"kubernetes.io/projected/2d2a55d8-22f4-48cd-8112-59c9328b9e05-kube-api-access-562hw\") pod \"ingress-canary-zkswt\" (UID: \"2d2a55d8-22f4-48cd-8112-59c9328b9e05\") " pod="openshift-ingress-canary/ingress-canary-zkswt"
Sep 11 19:19:09.555447 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.555372    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5eb3f3d-bd85-4887-8f0a-3173892dac83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.555447 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.555390    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpplq\" (UniqueName: \"kubernetes.io/projected/7e4162f7-09dd-44d1-85dd-d1380adfadef-kube-api-access-kpplq\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.569526 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.569497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:09.656324 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.656226    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxtwt\" (UniqueName: \"kubernetes.io/projected/c5eb3f3d-bd85-4887-8f0a-3173892dac83-kube-api-access-zxtwt\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.656324 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.656277    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5eb3f3d-bd85-4887-8f0a-3173892dac83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.656324 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.656325    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5eb3f3d-bd85-4887-8f0a-3173892dac83-data-volume\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.656561 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.656341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4162f7-09dd-44d1-85dd-d1380adfadef-tmp-dir\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.656561 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.656360    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e4162f7-09dd-44d1-85dd-d1380adfadef-metrics-tls\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.656561 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.656391    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5eb3f3d-bd85-4887-8f0a-3173892dac83-crio-socket\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.656561 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.656418    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-562hw\" (UniqueName: \"kubernetes.io/projected/2d2a55d8-22f4-48cd-8112-59c9328b9e05-kube-api-access-562hw\") pod \"ingress-canary-zkswt\" (UID: \"2d2a55d8-22f4-48cd-8112-59c9328b9e05\") " pod="openshift-ingress-canary/ingress-canary-zkswt"
Sep 11 19:19:09.656561 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.656441    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5eb3f3d-bd85-4887-8f0a-3173892dac83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.656561 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.656522    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5eb3f3d-bd85-4887-8f0a-3173892dac83-crio-socket\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.657077 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.657054    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4162f7-09dd-44d1-85dd-d1380adfadef-tmp-dir\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.657077 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.657070    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5eb3f3d-bd85-4887-8f0a-3173892dac83-data-volume\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.657261 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.657208    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpplq\" (UniqueName: \"kubernetes.io/projected/7e4162f7-09dd-44d1-85dd-d1380adfadef-kube-api-access-kpplq\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.657324 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.657313    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d2a55d8-22f4-48cd-8112-59c9328b9e05-cert\") pod \"ingress-canary-zkswt\" (UID: \"2d2a55d8-22f4-48cd-8112-59c9328b9e05\") " pod="openshift-ingress-canary/ingress-canary-zkswt"
Sep 11 19:19:09.657518 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.657490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4162f7-09dd-44d1-85dd-d1380adfadef-config-volume\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.657598 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.657535    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5eb3f3d-bd85-4887-8f0a-3173892dac83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.658447 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.658331    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4162f7-09dd-44d1-85dd-d1380adfadef-config-volume\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.661984 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.661960    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e4162f7-09dd-44d1-85dd-d1380adfadef-metrics-tls\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.662144 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.662122    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d2a55d8-22f4-48cd-8112-59c9328b9e05-cert\") pod \"ingress-canary-zkswt\" (UID: \"2d2a55d8-22f4-48cd-8112-59c9328b9e05\") " pod="openshift-ingress-canary/ingress-canary-zkswt"
Sep 11 19:19:09.662766 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.662740    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5eb3f3d-bd85-4887-8f0a-3173892dac83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.664288 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.664261    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-562hw\" (UniqueName: \"kubernetes.io/projected/2d2a55d8-22f4-48cd-8112-59c9328b9e05-kube-api-access-562hw\") pod \"ingress-canary-zkswt\" (UID: \"2d2a55d8-22f4-48cd-8112-59c9328b9e05\") " pod="openshift-ingress-canary/ingress-canary-zkswt"
Sep 11 19:19:09.664364 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.664353    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxtwt\" (UniqueName: \"kubernetes.io/projected/c5eb3f3d-bd85-4887-8f0a-3173892dac83-kube-api-access-zxtwt\") pod \"insights-runtime-extractor-5jj8q\" (UID: \"c5eb3f3d-bd85-4887-8f0a-3173892dac83\") " pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.664554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.664537    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpplq\" (UniqueName: \"kubernetes.io/projected/7e4162f7-09dd-44d1-85dd-d1380adfadef-kube-api-access-kpplq\") pod \"dns-default-49ldf\" (UID: \"7e4162f7-09dd-44d1-85dd-d1380adfadef\") " pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:09.670869 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.670842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:09.767173 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.767134    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zkswt"
Sep 11 19:19:09.770339 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.770310    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5jj8q"
Sep 11 19:19:09.770440 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.770422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:09.788285 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:09.788255    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:10.058711 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.058630    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5jj8q"]
Sep 11 19:19:10.061567 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.061537    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zkswt"]
Sep 11 19:19:10.062761 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:10.062742    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5eb3f3d_bd85_4887_8f0a_3173892dac83.slice/crio-bb4365cc10a1ea64650f3124115a555bbd11e0791ce2a79e3ebae80fc06a7274 WatchSource:0}: Error finding container bb4365cc10a1ea64650f3124115a555bbd11e0791ce2a79e3ebae80fc06a7274: Status 404 returned error can't find the container with id bb4365cc10a1ea64650f3124115a555bbd11e0791ce2a79e3ebae80fc06a7274
Sep 11 19:19:10.064592 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:10.064563    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d2a55d8_22f4_48cd_8112_59c9328b9e05.slice/crio-190665d41b2e5363763433d5eb2980a9d4e14aa842363a576fcfb4439a3bc2d8 WatchSource:0}: Error finding container 190665d41b2e5363763433d5eb2980a9d4e14aa842363a576fcfb4439a3bc2d8: Status 404 returned error can't find the container with id 190665d41b2e5363763433d5eb2980a9d4e14aa842363a576fcfb4439a3bc2d8
Sep 11 19:19:10.085657 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.085463    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-49ldf"]
Sep 11 19:19:10.088200 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:10.088175    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4162f7_09dd_44d1_85dd_d1380adfadef.slice/crio-e676d8b305760603cacf10424a3fe4dbb3a06c911952c7e6d97ebf92c4586210 WatchSource:0}: Error finding container e676d8b305760603cacf10424a3fe4dbb3a06c911952c7e6d97ebf92c4586210: Status 404 returned error can't find the container with id e676d8b305760603cacf10424a3fe4dbb3a06c911952c7e6d97ebf92c4586210
Sep 11 19:19:10.540520 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.540486    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:19:10.541144 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.541114    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:19:10.546178 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.545539    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 19:19:10.546178 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.546048    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 19:19:10.546396 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.546270    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ht26q\""
Sep 11 19:19:10.783309 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.783257    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:10.817012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.816981    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-49ldf" event={"ID":"7e4162f7-09dd-44d1-85dd-d1380adfadef","Type":"ContainerStarted","Data":"e676d8b305760603cacf10424a3fe4dbb3a06c911952c7e6d97ebf92c4586210"}
Sep 11 19:19:10.818853 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.818810    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5jj8q" event={"ID":"c5eb3f3d-bd85-4887-8f0a-3173892dac83","Type":"ContainerStarted","Data":"5f6027a28da625610caac2f3ab7ab17ed3c58bf5fb7d20fc26b65b6d4ec4bca8"}
Sep 11 19:19:10.818853 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.818854    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5jj8q" event={"ID":"c5eb3f3d-bd85-4887-8f0a-3173892dac83","Type":"ContainerStarted","Data":"bb4365cc10a1ea64650f3124115a555bbd11e0791ce2a79e3ebae80fc06a7274"}
Sep 11 19:19:10.820110 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.820081    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zkswt" event={"ID":"2d2a55d8-22f4-48cd-8112-59c9328b9e05","Type":"ContainerStarted","Data":"190665d41b2e5363763433d5eb2980a9d4e14aa842363a576fcfb4439a3bc2d8"}
Sep 11 19:19:10.976398 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:10.976362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:11.040656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:11.040563    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:11.229845 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:11.229807    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:11.825689 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:11.825650    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5jj8q" event={"ID":"c5eb3f3d-bd85-4887-8f0a-3173892dac83","Type":"ContainerStarted","Data":"b0abf0a385abc85937bb06dc97a314cea031ecf21277bb299c5f49a908f00f44"}
Sep 11 19:19:13.832901 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:13.832862    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-49ldf" event={"ID":"7e4162f7-09dd-44d1-85dd-d1380adfadef","Type":"ContainerStarted","Data":"cec4629b90524a972179e0842df6aaf78d6d61e93fc34d2190bdb026a817911e"}
Sep 11 19:19:13.833409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:13.832962    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-49ldf" event={"ID":"7e4162f7-09dd-44d1-85dd-d1380adfadef","Type":"ContainerStarted","Data":"d998e2ba5817c016f57389511de874060927b42253226da2a166922443f2b265"}
Sep 11 19:19:13.833409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:13.832993    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:13.834994 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:13.834960    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5jj8q" event={"ID":"c5eb3f3d-bd85-4887-8f0a-3173892dac83","Type":"ContainerStarted","Data":"2aa4ddb0469ec846edceca234b35b92c4e1e85748c3e90673b8d93fb03cf076e"}
Sep 11 19:19:13.836540 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:13.836517    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zkswt" event={"ID":"2d2a55d8-22f4-48cd-8112-59c9328b9e05","Type":"ContainerStarted","Data":"229907919da608c4b3a541906be90100ffcd0217f2c7ada51dac7c9b3bc04b95"}
Sep 11 19:19:13.848706 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:13.848651    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-49ldf" podStartSLOduration=2.124198465 podStartE2EDuration="4.848637723s" totalImagesPullingTime="2.724439258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:13.847393735 +0000 UTC m=+57.776018634" watchObservedRunningTime="2026-09-11 19:19:13.848637723 +0000 UTC m=+57.777262709"
Sep 11 19:19:13.862024 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:13.861981    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zkswt" podStartSLOduration=2.11410256 podStartE2EDuration="4.86196735s" totalImagesPullingTime="2.74786479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:13.861027076 +0000 UTC m=+57.789651987" watchObservedRunningTime="2026-09-11 19:19:13.86196735 +0000 UTC m=+57.790592244"
Sep 11 19:19:13.876660 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:13.876605    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5jj8q" podStartSLOduration=2.237655462 podStartE2EDuration="4.876586172s" totalImagesPullingTime="2.63893071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:19:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:13.876047027 +0000 UTC m=+57.804671928" watchObservedRunningTime="2026-09-11 19:19:13.876586172 +0000 UTC m=+57.805211074"
Sep 11 19:19:15.755356 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:15.755331    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfhvg"
Sep 11 19:19:15.903481 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:15.903451    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:15.955210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:15.955169    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:16.987286 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:16.987234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:17.047732 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:17.047702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:18.411104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.411072    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d8648b756-vqwlp"]
Sep 11 19:19:18.413544 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.413524    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.416107 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.416076    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 19:19:18.416225 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.416206    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 19:19:18.416974 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.416959    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ck5mr\""
Sep 11 19:19:18.417039 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.417008    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 19:19:18.417039 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.417020    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 19:19:18.417458 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.417445    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 19:19:18.418155 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.418142    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 19:19:18.420173 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.420154    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 19:19:18.432511 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.432492    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8648b756-vqwlp"]
Sep 11 19:19:18.519117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.519087    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-serving-cert\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.519117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.519119    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7hq2\" (UniqueName: \"kubernetes.io/projected/142d0475-389b-4d22-9f20-45f676380a7d-kube-api-access-q7hq2\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.519362 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.519189    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-console-config\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.519362 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.519210    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-oauth-config\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.519362 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.519228    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-service-ca\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.519470 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.519349    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-oauth-serving-cert\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.619853 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.619810    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-serving-cert\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.619853 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.619853    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7hq2\" (UniqueName: \"kubernetes.io/projected/142d0475-389b-4d22-9f20-45f676380a7d-kube-api-access-q7hq2\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.620093 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.619923    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-console-config\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.620093 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.619957    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-oauth-config\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.620093 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.619984    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-service-ca\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.620093 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.620024    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-oauth-serving-cert\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.620752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.620723    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-service-ca\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.620752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.620738    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-console-config\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.621046 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.621026    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-oauth-serving-cert\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.622637 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.622620    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-oauth-config\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.622700 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.622676    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-serving-cert\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.628141 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.628121    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7hq2\" (UniqueName: \"kubernetes.io/projected/142d0475-389b-4d22-9f20-45f676380a7d-kube-api-access-q7hq2\") pod \"console-6d8648b756-vqwlp\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") " pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.722880 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.722792    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:18.992901 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:18.992819    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8648b756-vqwlp"]
Sep 11 19:19:18.995555 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:18.995523    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142d0475_389b_4d22_9f20_45f676380a7d.slice/crio-6da9bab6a2cce76f41be79b70be8987440532fa80ade6a4afc451cda0a8ac088 WatchSource:0}: Error finding container 6da9bab6a2cce76f41be79b70be8987440532fa80ade6a4afc451cda0a8ac088: Status 404 returned error can't find the container with id 6da9bab6a2cce76f41be79b70be8987440532fa80ade6a4afc451cda0a8ac088
Sep 11 19:19:19.075064 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:19.075024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:19.175332 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:19.175264    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:19.233772 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:19.233744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:19.697121 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:19.697094    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:19.796910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:19.796854    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:19.856930 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:19.856899    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8648b756-vqwlp" event={"ID":"142d0475-389b-4d22-9f20-45f676380a7d","Type":"ContainerStarted","Data":"6da9bab6a2cce76f41be79b70be8987440532fa80ade6a4afc451cda0a8ac088"}
Sep 11 19:19:19.858144 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:19.858120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:22.249910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.249801    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:19:22.249910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.249893    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:19:22.252364 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.252344    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 19:19:22.253021 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.253005    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 19:19:22.262934 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.262906    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b7247e-b7a4-44db-a5dd-4817bb0df58a-metrics-certs\") pod \"network-metrics-daemon-dvg7l\" (UID: \"c1b7247e-b7a4-44db-a5dd-4817bb0df58a\") " pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:19:22.263032 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.262916    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2c8ebda-08bc-4433-9b6d-e4344b72ee3b-original-pull-secret\") pod \"global-pull-secret-syncer-69wfq\" (UID: \"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b\") " pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:19:22.451617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.451564    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5kl\" (UniqueName: \"kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl\") pod \"network-check-target-m6tdc\" (UID: \"c9378a65-f0bc-4a82-863c-a1647b854dd7\") " pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:19:22.453905 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.453885    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 19:19:22.464236 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.464215    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 19:19:22.475320 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.475299    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s5kl\" (UniqueName: \"kubernetes.io/projected/c9378a65-f0bc-4a82-863c-a1647b854dd7-kube-api-access-6s5kl\") pod \"network-check-target-m6tdc\" (UID: \"c9378a65-f0bc-4a82-863c-a1647b854dd7\") " pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:19:22.555237 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.555163    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ht26q\""
Sep 11 19:19:22.558951 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.558937    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69wfq"
Sep 11 19:19:22.563688 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.563671    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvg7l"
Sep 11 19:19:22.670446 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.670417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:22.729899 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.729866    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:22.750565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.750535    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tdnq7\""
Sep 11 19:19:22.759480 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.759457    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:19:22.791138 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.791104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:22.829796 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.829768    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-69wfq"]
Sep 11 19:19:22.832980 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:22.832940    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c8ebda_08bc_4433_9b6d_e4344b72ee3b.slice/crio-172bc30da2e7fd0db9378c367876a3d05fe421a4962fa3e5a21125c7a1187427 WatchSource:0}: Error finding container 172bc30da2e7fd0db9378c367876a3d05fe421a4962fa3e5a21125c7a1187427: Status 404 returned error can't find the container with id 172bc30da2e7fd0db9378c367876a3d05fe421a4962fa3e5a21125c7a1187427
Sep 11 19:19:22.834029 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.834005    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dvg7l"]
Sep 11 19:19:22.836822 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:22.836788    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b7247e_b7a4_44db_a5dd_4817bb0df58a.slice/crio-7ce7657344c24ea4fff2228425783903cc7b3eb9c3e4aa67de6153af53b8e66c WatchSource:0}: Error finding container 7ce7657344c24ea4fff2228425783903cc7b3eb9c3e4aa67de6153af53b8e66c: Status 404 returned error can't find the container with id 7ce7657344c24ea4fff2228425783903cc7b3eb9c3e4aa67de6153af53b8e66c
Sep 11 19:19:22.867116 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.867079    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-69wfq" event={"ID":"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b","Type":"ContainerStarted","Data":"172bc30da2e7fd0db9378c367876a3d05fe421a4962fa3e5a21125c7a1187427"}
Sep 11 19:19:22.868331 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.868301    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvg7l" event={"ID":"c1b7247e-b7a4-44db-a5dd-4817bb0df58a","Type":"ContainerStarted","Data":"7ce7657344c24ea4fff2228425783903cc7b3eb9c3e4aa67de6153af53b8e66c"}
Sep 11 19:19:22.869857 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.869837    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8648b756-vqwlp" event={"ID":"142d0475-389b-4d22-9f20-45f676380a7d","Type":"ContainerStarted","Data":"a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a"}
Sep 11 19:19:22.887401 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.887344    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d8648b756-vqwlp" podStartSLOduration=1.991109714 podStartE2EDuration="4.887324784s" totalImagesPullingTime="2.89621507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:22.886443098 +0000 UTC m=+66.815068008" watchObservedRunningTime="2026-09-11 19:19:22.887324784 +0000 UTC m=+66.815949681"
Sep 11 19:19:22.891446 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:22.891423    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:23.026611 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:23.026579    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m6tdc"]
Sep 11 19:19:23.029590 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:23.029564    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9378a65_f0bc_4a82_863c_a1647b854dd7.slice/crio-c74fedd4e09352346248be49e9662c0301cffbea849315efa7c4734dfb7fa0a5 WatchSource:0}: Error finding container c74fedd4e09352346248be49e9662c0301cffbea849315efa7c4734dfb7fa0a5: Status 404 returned error can't find the container with id c74fedd4e09352346248be49e9662c0301cffbea849315efa7c4734dfb7fa0a5
Sep 11 19:19:23.842812 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:23.842202    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-49ldf"
Sep 11 19:19:23.876326 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:23.876287    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m6tdc" event={"ID":"c9378a65-f0bc-4a82-863c-a1647b854dd7","Type":"ContainerStarted","Data":"c74fedd4e09352346248be49e9662c0301cffbea849315efa7c4734dfb7fa0a5"}
Sep 11 19:19:24.882039 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:24.881996    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvg7l" event={"ID":"c1b7247e-b7a4-44db-a5dd-4817bb0df58a","Type":"ContainerStarted","Data":"d9e1b2173019c342c9992bfd265013693ca6aeea90d0b4d861958f606e95b9d4"}
Sep 11 19:19:24.882523 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:24.882054    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvg7l" event={"ID":"c1b7247e-b7a4-44db-a5dd-4817bb0df58a","Type":"ContainerStarted","Data":"6a56e3ba18f904f72811d7af9208faea4a692952b26f8e706241a07c18cd570d"}
Sep 11 19:19:24.899017 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:24.898967    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dvg7l" podStartSLOduration=67.652247651 podStartE2EDuration="1m8.898951302s" totalImagesPullingTime="1.246703651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:24.898268526 +0000 UTC m=+68.826893419" watchObservedRunningTime="2026-09-11 19:19:24.898951302 +0000 UTC m=+68.827576224"
Sep 11 19:19:25.018452 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:25.018408    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:25.072436 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:25.072396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:25.574527 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:25.574489    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:25.685874 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:25.685837    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:26.588014 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:26.587969    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:26.685871 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:26.685838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:27.186267 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.186212    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-96fc5559-zw5dg"]
Sep 11 19:19:27.189343 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.189317    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.198468 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.198440    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-96fc5559-zw5dg"]
Sep 11 19:19:27.198622 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.198595    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 19:19:27.289201 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.289159    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-service-ca\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.289407 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.289214    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-config\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.289407 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.289260    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-oauth-config\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.289407 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.289321    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-trusted-ca-bundle\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.289407 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.289348    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-oauth-serving-cert\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.289407 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.289377    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcdnr\" (UniqueName: \"kubernetes.io/projected/5b169eaa-b951-4980-b7ff-0d4adeb818d6-kube-api-access-xcdnr\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.289407 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.289398    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-serving-cert\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.390203 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.390161    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-service-ca\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.390413 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.390212    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-config\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.390413 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.390286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-oauth-config\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.390413 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.390339    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-trusted-ca-bundle\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.390413 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.390365    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-oauth-serving-cert\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.390413 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.390396    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcdnr\" (UniqueName: \"kubernetes.io/projected/5b169eaa-b951-4980-b7ff-0d4adeb818d6-kube-api-access-xcdnr\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.390672 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.390434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-serving-cert\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.391128 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.391055    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-config\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.391259 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.391196    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-service-ca\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.391379 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.391357    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-trusted-ca-bundle\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.391878 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.391857    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-oauth-serving-cert\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.393050 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.393031    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-oauth-config\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.393163 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.393143    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-serving-cert\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.399644 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.399617    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcdnr\" (UniqueName: \"kubernetes.io/projected/5b169eaa-b951-4980-b7ff-0d4adeb818d6-kube-api-access-xcdnr\") pod \"console-96fc5559-zw5dg\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") " pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.501101 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.500811    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:27.565927 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.565892    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:27.618757 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.618724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:27.894065 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.894026    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-69wfq" event={"ID":"c2c8ebda-08bc-4433-9b6d-e4344b72ee3b","Type":"ContainerStarted","Data":"a8176cded41bc816af565d72402dfa8c18e6c2397d9acba69fe158e9b9b59eda"}
Sep 11 19:19:27.895650 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.895624    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m6tdc" event={"ID":"c9378a65-f0bc-4a82-863c-a1647b854dd7","Type":"ContainerStarted","Data":"0a623ecdbf5f8296897da4cdfc8cfcf0e6e689a3e5469cf751e623f8edb718b9"}
Sep 11 19:19:27.895784 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.895756    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:19:27.909222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.909162    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-69wfq" podStartSLOduration=67.074191998 podStartE2EDuration="1m11.909141915s" totalImagesPullingTime="4.834949917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:27.908367612 +0000 UTC m=+71.836992511" watchObservedRunningTime="2026-09-11 19:19:27.909141915 +0000 UTC m=+71.837766816"
Sep 11 19:19:27.924534 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.924491    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m6tdc" podStartSLOduration=67.294802383 podStartE2EDuration="1m11.924476942s" totalImagesPullingTime="4.629674559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:27.923911883 +0000 UTC m=+71.852536783" watchObservedRunningTime="2026-09-11 19:19:27.924476942 +0000 UTC m=+71.853101840"
Sep 11 19:19:27.934152 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:27.934123    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-96fc5559-zw5dg"]
Sep 11 19:19:27.937124 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:27.937099    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b169eaa_b951_4980_b7ff_0d4adeb818d6.slice/crio-f929e9a22caf898383e4bd1707f91b1c5632fc9a17c1a7dbef23f6e0a39a7d8c WatchSource:0}: Error finding container f929e9a22caf898383e4bd1707f91b1c5632fc9a17c1a7dbef23f6e0a39a7d8c: Status 404 returned error can't find the container with id f929e9a22caf898383e4bd1707f91b1c5632fc9a17c1a7dbef23f6e0a39a7d8c
Sep 11 19:19:28.596372 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:28.596324    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:28.654735 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:28.654689    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:28.723468 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:28.723421    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:28.723661 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:28.723508    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:28.728205 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:28.728184    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:28.900828 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:28.900727    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96fc5559-zw5dg" event={"ID":"5b169eaa-b951-4980-b7ff-0d4adeb818d6","Type":"ContainerStarted","Data":"e43d0acfa017bcb8f5e4dbfa73cc804017e8c5fce57ac14bc9beb1b9ec1dd196"}
Sep 11 19:19:28.900828 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:28.900779    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96fc5559-zw5dg" event={"ID":"5b169eaa-b951-4980-b7ff-0d4adeb818d6","Type":"ContainerStarted","Data":"f929e9a22caf898383e4bd1707f91b1c5632fc9a17c1a7dbef23f6e0a39a7d8c"}
Sep 11 19:19:28.904910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:28.904890    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:19:28.919853 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:28.919808    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-96fc5559-zw5dg" podStartSLOduration=1.91979473 podStartE2EDuration="1.91979473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:28.918879867 +0000 UTC m=+72.847504765" watchObservedRunningTime="2026-09-11 19:19:28.91979473 +0000 UTC m=+72.848419629"
Sep 11 19:19:29.991576 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:29.991543    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:30.862787 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:30.862748    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:30.941082 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:30.941049    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:31.039997 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:31.039970    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:31.800300 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:31.800264    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:32.200732 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.200701    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:32.411352 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.411319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:32.425673 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.425650    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"]
Sep 11 19:19:32.428869 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.428855    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.431195 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.431175    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 19:19:32.431948 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.431891    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 19:19:32.431948 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.431908    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 19:19:32.431948 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.431891    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 19:19:32.431948 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.431930    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 19:19:32.432166 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.431892    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qkh45\""
Sep 11 19:19:32.443050 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.443029    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"]
Sep 11 19:19:32.455952 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.455877    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"]
Sep 11 19:19:32.459975 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.459958    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.465341 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.464407    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 11 19:19:32.465341 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.464634    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fgz7x\""
Sep 11 19:19:32.465341 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.464869    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 11 19:19:32.465341 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.465152    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 11 19:19:32.473671 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.473651    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8sx7t"]
Sep 11 19:19:32.476678 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.476661    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.479157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.479123    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 19:19:32.479278 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.479166    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6zjdh\""
Sep 11 19:19:32.479468 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.479450    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 19:19:32.479662 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.479640    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 19:19:32.480081 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.480064    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"]
Sep 11 19:19:32.512489 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.512458    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:32.526356 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526326    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.526488 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526362    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c94c623-37c1-45d1-83fa-c2663aff05ac-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.526488 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526390    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdhmn\" (UniqueName: \"kubernetes.io/projected/1c94c623-37c1-45d1-83fa-c2663aff05ac-kube-api-access-fdhmn\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.526488 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526450    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbqft\" (UniqueName: \"kubernetes.io/projected/dcb322b1-5921-4762-a0e7-f520adef0774-kube-api-access-pbqft\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.526608 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526505    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.526608 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526526    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c94c623-37c1-45d1-83fa-c2663aff05ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.526608 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.526608 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526570    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.526736 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526618    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-wtmp\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.526736 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526647    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.526736 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526673    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c94c623-37c1-45d1-83fa-c2663aff05ac-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.526736 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526721    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzjbh\" (UniqueName: \"kubernetes.io/projected/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-api-access-bzjbh\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.526895 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.526895 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526794    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcb322b1-5921-4762-a0e7-f520adef0774-metrics-client-ca\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.526895 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526818    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-tls\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.526895 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526843    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcb322b1-5921-4762-a0e7-f520adef0774-sys\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.526895 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526859    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.526895 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526882    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dcb322b1-5921-4762-a0e7-f520adef0774-root\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.527094 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.526895    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-textfile\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.610648 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.610610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:32.627768 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.627736    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.627768 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.627768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcb322b1-5921-4762-a0e7-f520adef0774-metrics-client-ca\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.627988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.627794    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-tls\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.627988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.627820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcb322b1-5921-4762-a0e7-f520adef0774-sys\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.627988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.627871    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcb322b1-5921-4762-a0e7-f520adef0774-sys\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.627988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.627907    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.627988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.627933    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dcb322b1-5921-4762-a0e7-f520adef0774-root\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.627988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.627955    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-textfile\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.628305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628013    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dcb322b1-5921-4762-a0e7-f520adef0774-root\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.628305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628093    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.628305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628139    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c94c623-37c1-45d1-83fa-c2663aff05ac-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.628305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628173    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdhmn\" (UniqueName: \"kubernetes.io/projected/1c94c623-37c1-45d1-83fa-c2663aff05ac-kube-api-access-fdhmn\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.628305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbqft\" (UniqueName: \"kubernetes.io/projected/dcb322b1-5921-4762-a0e7-f520adef0774-kube-api-access-pbqft\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.628305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628234    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.628305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628285    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c94c623-37c1-45d1-83fa-c2663aff05ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628322    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628353    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628378    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-wtmp\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628405    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628436    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c94c623-37c1-45d1-83fa-c2663aff05ac-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628462    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-textfile\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628474    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcb322b1-5921-4762-a0e7-f520adef0774-metrics-client-ca\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628464    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzjbh\" (UniqueName: \"kubernetes.io/projected/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-api-access-bzjbh\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628556    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:32.628569    2583 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628578    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.628640 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:19:32.628636    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c94c623-37c1-45d1-83fa-c2663aff05ac-openshift-state-metrics-tls podName:1c94c623-37c1-45d1-83fa-c2663aff05ac nodeName:}" failed. No retries permitted until 2026-09-11 19:19:33.128616815 +0000 UTC m=+77.057241705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1c94c623-37c1-45d1-83fa-c2663aff05ac-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-tbrkv" (UID: "1c94c623-37c1-45d1-83fa-c2663aff05ac") : secret "openshift-state-metrics-tls" not found
Sep 11 19:19:32.629187 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.628871    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-wtmp\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.629187 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.629034    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.629332 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.629310    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c94c623-37c1-45d1-83fa-c2663aff05ac-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.629476 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.629406    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.631089 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.631064    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.631205 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.631129    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dcb322b1-5921-4762-a0e7-f520adef0774-node-exporter-tls\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.631386 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.631368    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.631447 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.631379    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.631664 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.631650    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c94c623-37c1-45d1-83fa-c2663aff05ac-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.639151 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.639129    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbqft\" (UniqueName: \"kubernetes.io/projected/dcb322b1-5921-4762-a0e7-f520adef0774-kube-api-access-pbqft\") pod \"node-exporter-8sx7t\" (UID: \"dcb322b1-5921-4762-a0e7-f520adef0774\") " pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.639885 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.639865    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdhmn\" (UniqueName: \"kubernetes.io/projected/1c94c623-37c1-45d1-83fa-c2663aff05ac-kube-api-access-fdhmn\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:32.639979 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.639871    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzjbh\" (UniqueName: \"kubernetes.io/projected/b4168c7b-8b2a-44d6-a77e-896e0a65b4fb-kube-api-access-bzjbh\") pod \"kube-state-metrics-59fbdd77d-tf26r\" (UID: \"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.708197 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.708120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:32.773491 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.773450    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"
Sep 11 19:19:32.785959 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.785925    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8sx7t"
Sep 11 19:19:32.799946 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:32.799916    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcb322b1_5921_4762_a0e7_f520adef0774.slice/crio-e20cec6c45ce0c7c86723f33fc0e43f250d3df00f1cadb1c31ea882146c9b3a1 WatchSource:0}: Error finding container e20cec6c45ce0c7c86723f33fc0e43f250d3df00f1cadb1c31ea882146c9b3a1: Status 404 returned error can't find the container with id e20cec6c45ce0c7c86723f33fc0e43f250d3df00f1cadb1c31ea882146c9b3a1
Sep 11 19:19:32.808462 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.808436    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:32.909062 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.909031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:32.919799 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:32.919765    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sx7t" event={"ID":"dcb322b1-5921-4762-a0e7-f520adef0774","Type":"ContainerStarted","Data":"e20cec6c45ce0c7c86723f33fc0e43f250d3df00f1cadb1c31ea882146c9b3a1"}
Sep 11 19:19:33.009486 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.009376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:33.049282 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.049261    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r"]
Sep 11 19:19:33.051615 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:33.051591    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4168c7b_8b2a_44d6_a77e_896e0a65b4fb.slice/crio-0677ea8162826ee659f8a2dcb65d96a0c914a653cd1401318acd43068359f043 WatchSource:0}: Error finding container 0677ea8162826ee659f8a2dcb65d96a0c914a653cd1401318acd43068359f043: Status 404 returned error can't find the container with id 0677ea8162826ee659f8a2dcb65d96a0c914a653cd1401318acd43068359f043
Sep 11 19:19:33.110157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.110123    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:33.133320 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.133290    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c94c623-37c1-45d1-83fa-c2663aff05ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:33.135823 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.135804    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c94c623-37c1-45d1-83fa-c2663aff05ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-tbrkv\" (UID: \"1c94c623-37c1-45d1-83fa-c2663aff05ac\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:33.338213 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.338124    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"
Sep 11 19:19:33.419279 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.419201    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:33.510965 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.510933    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:33.619320 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.619293    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv"]
Sep 11 19:19:33.622508 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:33.622477    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c94c623_37c1_45d1_83fa_c2663aff05ac.slice/crio-847de27c803c567f707cef2265cfd2a4ea8a338fc229187293a09805ef3fc396 WatchSource:0}: Error finding container 847de27c803c567f707cef2265cfd2a4ea8a338fc229187293a09805ef3fc396: Status 404 returned error can't find the container with id 847de27c803c567f707cef2265cfd2a4ea8a338fc229187293a09805ef3fc396
Sep 11 19:19:33.753324 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.752729    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:33.926511 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.926433    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv" event={"ID":"1c94c623-37c1-45d1-83fa-c2663aff05ac","Type":"ContainerStarted","Data":"af3fb13e671387f4ca3a312ec3b94ebd414f1cbcf9ae937a3181059088ee7274"}
Sep 11 19:19:33.926511 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.926505    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv" event={"ID":"1c94c623-37c1-45d1-83fa-c2663aff05ac","Type":"ContainerStarted","Data":"dff714fbc8c746b81a664169f580dc515cd00650f483df513677ddf2e9a1ac83"}
Sep 11 19:19:33.926761 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.926534    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv" event={"ID":"1c94c623-37c1-45d1-83fa-c2663aff05ac","Type":"ContainerStarted","Data":"847de27c803c567f707cef2265cfd2a4ea8a338fc229187293a09805ef3fc396"}
Sep 11 19:19:33.927978 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:33.927940    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r" event={"ID":"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb","Type":"ContainerStarted","Data":"0677ea8162826ee659f8a2dcb65d96a0c914a653cd1401318acd43068359f043"}
Sep 11 19:19:34.501065 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.501029    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"]
Sep 11 19:19:34.507665 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.507641    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.512494 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.511067    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 19:19:34.512494 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.511615    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-du95i4o1kdpke\""
Sep 11 19:19:34.512494 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.511865    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 19:19:34.512494 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.512064    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 19:19:34.512494 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.512329    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 19:19:34.512494 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.512378    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-n4mv7\""
Sep 11 19:19:34.512494 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.512390    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 19:19:34.520087 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.520056    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"]
Sep 11 19:19:34.648117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.648089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-tls\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.648402 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.648125    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.648402 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.648161    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.648402 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.648190    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-grpc-tls\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.648402 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.648230    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.648402 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.648300    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.648402 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.648321    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5tqb\" (UniqueName: \"kubernetes.io/projected/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-kube-api-access-x5tqb\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.648402 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.648359    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-metrics-client-ca\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.749497 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.749458    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.749497 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.749506    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5tqb\" (UniqueName: \"kubernetes.io/projected/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-kube-api-access-x5tqb\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.749782 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.749536    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-metrics-client-ca\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.749782 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.749585    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-tls\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.749782 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.749613    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.749782 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.749646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.749782 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.749754    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-grpc-tls\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.750012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.749825    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.750681 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.750648    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-metrics-client-ca\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.752565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.752541    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.753015 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.752993    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.753015 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.753010    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.753118 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.753058    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-grpc-tls\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.753484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.753464    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.753586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.753538    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-secret-thanos-querier-tls\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.760174 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.760152    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5tqb\" (UniqueName: \"kubernetes.io/projected/a13f0d80-5d86-4ad0-aee7-1809b1b69cce-kube-api-access-x5tqb\") pod \"thanos-querier-7b6c8c8556-xzzlq\" (UID: \"a13f0d80-5d86-4ad0-aee7-1809b1b69cce\") " pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.875419 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.875387    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:34.934497 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.934406    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r" event={"ID":"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb","Type":"ContainerStarted","Data":"8ad0d4b1c79140062fd86c65ceb9e3177cfb2e5533908aa871745bb335105b6d"}
Sep 11 19:19:34.934673 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.934495    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r" event={"ID":"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb","Type":"ContainerStarted","Data":"6dac750cd93885627a9eb21188cceeee0476a0afc2c06ead4d7e180fd93b34b3"}
Sep 11 19:19:34.934673 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.934529    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r" event={"ID":"b4168c7b-8b2a-44d6-a77e-896e0a65b4fb","Type":"ContainerStarted","Data":"27b46e01f38c5edf976b65d55dd177f3dec48dda78c4a2318e91f693167bb8e4"}
Sep 11 19:19:34.936691 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.936584    2583 generic.go:356] "Generic (PLEG): container finished" podID="dcb322b1-5921-4762-a0e7-f520adef0774" containerID="f3cd408454c66ea5b0ed762ea3d3c7848660bd839800ad589fba1c6004b4b590" exitCode=0
Sep 11 19:19:34.936691 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.936638    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sx7t" event={"ID":"dcb322b1-5921-4762-a0e7-f520adef0774","Type":"ContainerDied","Data":"f3cd408454c66ea5b0ed762ea3d3c7848660bd839800ad589fba1c6004b4b590"}
Sep 11 19:19:34.953309 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:34.953228    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tf26r" podStartSLOduration=1.665289031 podStartE2EDuration="2.953204344s" totalImagesPullingTime="1.287915313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:34.952335119 +0000 UTC m=+78.880960016" watchObservedRunningTime="2026-09-11 19:19:34.953204344 +0000 UTC m=+78.881829245"
Sep 11 19:19:35.145663 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.145373    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:35.241236 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.241144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:35.343618 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.343594    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"]
Sep 11 19:19:35.345807 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:35.345783    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13f0d80_5d86_4ad0_aee7_1809b1b69cce.slice/crio-f3b382aac91dc4bd60228baf9d723fc8551a152aa1b2faab2c2c0f52d178684a WatchSource:0}: Error finding container f3b382aac91dc4bd60228baf9d723fc8551a152aa1b2faab2c2c0f52d178684a: Status 404 returned error can't find the container with id f3b382aac91dc4bd60228baf9d723fc8551a152aa1b2faab2c2c0f52d178684a
Sep 11 19:19:35.400340 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.400311    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:35.500886 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.500807    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:35.601133 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.601103    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:35.719290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.719235    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:35.942005 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.941967    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sx7t" event={"ID":"dcb322b1-5921-4762-a0e7-f520adef0774","Type":"ContainerStarted","Data":"cbed4b22864cd45c89277c91dfbeb2c5af9347771d8e936f41b4b38420961cd4"}
Sep 11 19:19:35.942205 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.942029    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sx7t" event={"ID":"dcb322b1-5921-4762-a0e7-f520adef0774","Type":"ContainerStarted","Data":"f1b08641e18cd52abcd814dbe9d9fcdbdf5fb308c120d93ace30b9171f1717c2"}
Sep 11 19:19:35.943223 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.943202    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq" event={"ID":"a13f0d80-5d86-4ad0-aee7-1809b1b69cce","Type":"ContainerStarted","Data":"f3b382aac91dc4bd60228baf9d723fc8551a152aa1b2faab2c2c0f52d178684a"}
Sep 11 19:19:35.945293 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.945270    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv" event={"ID":"1c94c623-37c1-45d1-83fa-c2663aff05ac","Type":"ContainerStarted","Data":"4760bf9d168e468f395b224e908e2653116f4e96eeb68508d8826162567ee03d"}
Sep 11 19:19:35.967859 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.967801    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8sx7t" podStartSLOduration=2.336941445 podStartE2EDuration="3.967780589s" totalImagesPullingTime="1.537582128s" totalInitContainerRuntime="93.257016ms" isStatefulPod=true podCreationTimestamp="2026-09-11 19:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:35.966302694 +0000 UTC m=+79.894927595" watchObservedRunningTime="2026-09-11 19:19:35.967780589 +0000 UTC m=+79.896405488"
Sep 11 19:19:35.984166 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:35.984119    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-tbrkv" podStartSLOduration=2.6374822719999997 podStartE2EDuration="3.9841035s" totalImagesPullingTime="1.346621228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:35.983510526 +0000 UTC m=+79.912135417" watchObservedRunningTime="2026-09-11 19:19:35.9841035 +0000 UTC m=+79.912728397"
Sep 11 19:19:36.599912 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.599882    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:36.695597 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.695559    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:36.783824 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.783789    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-69d66b8655-mnbcs"]
Sep 11 19:19:36.787069 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.787044    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.789493 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.789460    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 19:19:36.789805 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.789784    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 19:19:36.790973 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.790953    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-b332ot6pje2l8\""
Sep 11 19:19:36.791057 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.790976    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 19:19:36.791057 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.791033    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-mzzrz\""
Sep 11 19:19:36.791165 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.790951    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 19:19:36.797434 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.797408    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69d66b8655-mnbcs"]
Sep 11 19:19:36.871621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.871536    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9193ded0-a7a6-4bd8-bd27-78407889a914-client-ca-bundle\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.871621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.871573    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9193ded0-a7a6-4bd8-bd27-78407889a914-secret-metrics-server-tls\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.871621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.871600    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9193ded0-a7a6-4bd8-bd27-78407889a914-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.871881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.871715    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9193ded0-a7a6-4bd8-bd27-78407889a914-audit-log\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.871881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.871751    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9193ded0-a7a6-4bd8-bd27-78407889a914-secret-metrics-server-client-certs\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.871881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.871776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsbgv\" (UniqueName: \"kubernetes.io/projected/9193ded0-a7a6-4bd8-bd27-78407889a914-kube-api-access-wsbgv\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.871881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.871848    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9193ded0-a7a6-4bd8-bd27-78407889a914-metrics-server-audit-profiles\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.972311 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.972269    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9193ded0-a7a6-4bd8-bd27-78407889a914-secret-metrics-server-tls\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.972493 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.972327    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9193ded0-a7a6-4bd8-bd27-78407889a914-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.972493 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.972387    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9193ded0-a7a6-4bd8-bd27-78407889a914-audit-log\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.972595 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.972507    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9193ded0-a7a6-4bd8-bd27-78407889a914-secret-metrics-server-client-certs\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.972595 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.972538    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsbgv\" (UniqueName: \"kubernetes.io/projected/9193ded0-a7a6-4bd8-bd27-78407889a914-kube-api-access-wsbgv\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.972595 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.972559    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9193ded0-a7a6-4bd8-bd27-78407889a914-metrics-server-audit-profiles\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.972734 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.972627    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9193ded0-a7a6-4bd8-bd27-78407889a914-client-ca-bundle\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.973265 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.973174    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9193ded0-a7a6-4bd8-bd27-78407889a914-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.973742 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.973720    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9193ded0-a7a6-4bd8-bd27-78407889a914-metrics-server-audit-profiles\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.973886 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.973868    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9193ded0-a7a6-4bd8-bd27-78407889a914-audit-log\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.975341 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.975317    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9193ded0-a7a6-4bd8-bd27-78407889a914-secret-metrics-server-tls\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.975463 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.975440    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9193ded0-a7a6-4bd8-bd27-78407889a914-secret-metrics-server-client-certs\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.975576 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.975557    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9193ded0-a7a6-4bd8-bd27-78407889a914-client-ca-bundle\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:36.980497 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:36.980476    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsbgv\" (UniqueName: \"kubernetes.io/projected/9193ded0-a7a6-4bd8-bd27-78407889a914-kube-api-access-wsbgv\") pod \"metrics-server-69d66b8655-mnbcs\" (UID: \"9193ded0-a7a6-4bd8-bd27-78407889a914\") " pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:37.100138 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.100105    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:37.241817 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.241746    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:37.342450 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.342420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:37.501658 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.501575    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:37.503037 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.501733    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:37.508030 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.507978    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:37.515356 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.515328    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69d66b8655-mnbcs"]
Sep 11 19:19:37.518712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.518689    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:37.518971 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:37.518951    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9193ded0_a7a6_4bd8_bd27_78407889a914.slice/crio-de8eef5b53ae23c3c5247e983f9dfe1703e657965ac072868c3c0850984d2aac WatchSource:0}: Error finding container de8eef5b53ae23c3c5247e983f9dfe1703e657965ac072868c3c0850984d2aac: Status 404 returned error can't find the container with id de8eef5b53ae23c3c5247e983f9dfe1703e657965ac072868c3c0850984d2aac
Sep 11 19:19:37.681305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.681186    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:37.781000 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.780900    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:37.956467 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.956425    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq" event={"ID":"a13f0d80-5d86-4ad0-aee7-1809b1b69cce","Type":"ContainerStarted","Data":"67a95e610caaa7bf0f40c4b25a10fca93e2c24791b51cd80965569e7e4e5ed4b"}
Sep 11 19:19:37.956669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.956521    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq" event={"ID":"a13f0d80-5d86-4ad0-aee7-1809b1b69cce","Type":"ContainerStarted","Data":"ee1ac2637835efeba52ac09fa9168be8c69bc80a77271546ef93ea0c3e6048fb"}
Sep 11 19:19:37.956669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.956555    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq" event={"ID":"a13f0d80-5d86-4ad0-aee7-1809b1b69cce","Type":"ContainerStarted","Data":"be44ceef29fd349c02507a8a25be0f1efcc9b7473052bc9eb74ccee592585e17"}
Sep 11 19:19:37.957903 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.957868    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs" event={"ID":"9193ded0-a7a6-4bd8-bd27-78407889a914","Type":"ContainerStarted","Data":"de8eef5b53ae23c3c5247e983f9dfe1703e657965ac072868c3c0850984d2aac"}
Sep 11 19:19:37.963377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:37.963353    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:19:38.026612 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.026577    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d8648b756-vqwlp"]
Sep 11 19:19:38.796825 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.796788    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 19:19:38.800766 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.800742    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.804295 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.804261    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 19:19:38.804295 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.804276    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 19:19:38.804880 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.804853    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 19:19:38.806307 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.804861    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 19:19:38.806307 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.805124    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9p662s6vcma48\""
Sep 11 19:19:38.806307 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.805512    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 19:19:38.806307 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.805510    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7v5pw\""
Sep 11 19:19:38.806307 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.805744    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 19:19:38.806566 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.806403    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 19:19:38.806654 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.806634    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 19:19:38.806715 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.806694    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 19:19:38.809897 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.809874    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 19:19:38.811206 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.809877    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 19:19:38.811206 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.810157    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 19:19:38.811206 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.810894    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 19:19:38.812217 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.812197    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 19:19:38.816583 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.816560    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:38.839847 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.839819    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 19:19:38.889568 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889534    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.889747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889585    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.889747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889614    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94m9m\" (UniqueName: \"kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-kube-api-access-94m9m\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.889747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.889747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889653    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.889747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889677    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-config\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.889747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889721    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-config-out\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889781    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889820    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889839    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889853    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889879    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-web-config\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889948    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.889974    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.890000    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.890028    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.890072    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.890088    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.890694 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.890147    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.931563 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.931530    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:38.966869 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.966725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq" event={"ID":"a13f0d80-5d86-4ad0-aee7-1809b1b69cce","Type":"ContainerStarted","Data":"dbd35b73225757b2a1edf6d2201831e2018b9bb67fdcb10c323b068f10fd8a4d"}
Sep 11 19:19:38.966869 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.966816    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq" event={"ID":"a13f0d80-5d86-4ad0-aee7-1809b1b69cce","Type":"ContainerStarted","Data":"16261cb240d58a716793071d8998f98bbcf60ce0af3b53fb8eac1d111ab1304c"}
Sep 11 19:19:38.966869 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.966857    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq" event={"ID":"a13f0d80-5d86-4ad0-aee7-1809b1b69cce","Type":"ContainerStarted","Data":"14beab8aaed53e12a67b49af165b4c814e1c5f3da6b61378d6279dd0cc46337d"}
Sep 11 19:19:38.991164 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991129    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-config\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991339 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991172    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-config-out\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991339 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991339 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991234    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991339 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991282    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991339 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991306    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991339 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-web-config\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991365    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991393    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991421    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991445    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991531    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991555    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991577    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991607    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991918 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991638    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991918 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991658    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94m9m\" (UniqueName: \"kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-kube-api-access-94m9m\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991918 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991687    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.991918 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.991716    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.992916 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.992886    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.993022 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.992887    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.993539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.993513    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.994118 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.994095    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.994950 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.994926    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.996834 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.996201    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq" podStartSLOduration=2.247772475 podStartE2EDuration="4.996183475s" totalImagesPullingTime="2.748411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:19:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:38.995438232 +0000 UTC m=+82.924063159" watchObservedRunningTime="2026-09-11 19:19:38.996183475 +0000 UTC m=+82.924808375"
Sep 11 19:19:38.997436 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.997371    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.998857 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.998810    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:38.999966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:38.999941    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.000114 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.000095    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.000656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.000634    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.001369 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.000991    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-config\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.001369 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.001342    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-web-config\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.001517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.001291    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.001517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.001140    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.001517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.001385    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.001657 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.001582    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.002169 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.002141    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.002967 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.002923    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.003931 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.003911    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94m9m\" (UniqueName: \"kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-kube-api-access-94m9m\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.006697 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.006678    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-config-out\") pod \"prometheus-k8s-0\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.031826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.031794    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:39.115311 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.115288    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:39.398437 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.398000    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 19:19:39.400846 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:19:39.400778    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod385f5010_14f0_443d_b648_0354252fa18e.slice/crio-7d23e81be15234e87c10a4f519180bc43139741bf838c8df7abaa27f7fe8c80f WatchSource:0}: Error finding container 7d23e81be15234e87c10a4f519180bc43139741bf838c8df7abaa27f7fe8c80f: Status 404 returned error can't find the container with id 7d23e81be15234e87c10a4f519180bc43139741bf838c8df7abaa27f7fe8c80f
Sep 11 19:19:39.976281 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.975680    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs" event={"ID":"9193ded0-a7a6-4bd8-bd27-78407889a914","Type":"ContainerStarted","Data":"9f035f379562f68d7260aa56196ccae8ebafb0c181746f072994a8b7e241e326"}
Sep 11 19:19:39.979308 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.979277    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerStarted","Data":"7d23e81be15234e87c10a4f519180bc43139741bf838c8df7abaa27f7fe8c80f"}
Sep 11 19:19:39.979648 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.979627    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:39.993149 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:39.993095    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs" podStartSLOduration=2.426031813 podStartE2EDuration="3.993078552s" totalImagesPullingTime="1.567046739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:19:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:39.992685536 +0000 UTC m=+83.921310446" watchObservedRunningTime="2026-09-11 19:19:39.993078552 +0000 UTC m=+83.921703449"
Sep 11 19:19:40.055584 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:40.055539    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:40.156702 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:40.156666    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:40.984431 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:40.984390    2583 generic.go:356] "Generic (PLEG): container finished" podID="385f5010-14f0-443d-b648-0354252fa18e" containerID="222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018" exitCode=0
Sep 11 19:19:40.984906 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:40.984483    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerDied","Data":"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"}
Sep 11 19:19:45.010819 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:45.010448    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerStarted","Data":"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"}
Sep 11 19:19:45.010819 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:45.010518    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerStarted","Data":"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"}
Sep 11 19:19:45.010819 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:45.010544    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerStarted","Data":"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"}
Sep 11 19:19:45.990791 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:45.990762    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7b6c8c8556-xzzlq"
Sep 11 19:19:46.019709 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:46.019677    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerStarted","Data":"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"}
Sep 11 19:19:46.020084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:46.019735    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerStarted","Data":"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"}
Sep 11 19:19:46.020084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:46.019761    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerStarted","Data":"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"}
Sep 11 19:19:46.053628 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:46.053561    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.294566761 podStartE2EDuration="8.053536741s" totalImagesPullingTime="4.739023155s" totalInitContainerRuntime="19.946825ms" isStatefulPod=false podCreationTimestamp="2026-09-11 19:19:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:19:46.050507873 +0000 UTC m=+89.979132782" watchObservedRunningTime="2026-09-11 19:19:46.053536741 +0000 UTC m=+89.982161640"
Sep 11 19:19:49.116006 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:49.115958    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:19:49.991100 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:49.991069    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-96fc5559-zw5dg"]
Sep 11 19:19:50.112393 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:50.112351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:50.167076 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:50.167041    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:50.222539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:50.222514    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:52.673773 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:52.673740    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:52.772763 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:52.772720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:19:57.100488 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:57.100452    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:57.100873 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:57.100515    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:19:58.903596 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:58.903567    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m6tdc"
Sep 11 19:19:59.579151 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:19:59.579116    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:20:03.046516 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.046475    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d8648b756-vqwlp" podUID="142d0475-389b-4d22-9f20-45f676380a7d" containerName="console" containerID="cri-o://a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a" gracePeriod=15
Sep 11 19:20:03.289555 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.289532    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:20:03.310526 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.310439    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-oauth-serving-cert\") pod \"142d0475-389b-4d22-9f20-45f676380a7d\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") "
Sep 11 19:20:03.310526 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.310509    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-console-config\") pod \"142d0475-389b-4d22-9f20-45f676380a7d\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") "
Sep 11 19:20:03.310740 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.310545    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-oauth-config\") pod \"142d0475-389b-4d22-9f20-45f676380a7d\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") "
Sep 11 19:20:03.310740 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.310605    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7hq2\" (UniqueName: \"kubernetes.io/projected/142d0475-389b-4d22-9f20-45f676380a7d-kube-api-access-q7hq2\") pod \"142d0475-389b-4d22-9f20-45f676380a7d\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") "
Sep 11 19:20:03.310740 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.310647    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-serving-cert\") pod \"142d0475-389b-4d22-9f20-45f676380a7d\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") "
Sep 11 19:20:03.310740 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.310668    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-service-ca\") pod \"142d0475-389b-4d22-9f20-45f676380a7d\" (UID: \"142d0475-389b-4d22-9f20-45f676380a7d\") "
Sep 11 19:20:03.310944 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.310914    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "142d0475-389b-4d22-9f20-45f676380a7d" (UID: "142d0475-389b-4d22-9f20-45f676380a7d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:03.311012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.310919    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-console-config" (OuterVolumeSpecName: "console-config") pod "142d0475-389b-4d22-9f20-45f676380a7d" (UID: "142d0475-389b-4d22-9f20-45f676380a7d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:03.311205 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.311185    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-service-ca" (OuterVolumeSpecName: "service-ca") pod "142d0475-389b-4d22-9f20-45f676380a7d" (UID: "142d0475-389b-4d22-9f20-45f676380a7d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:03.313130 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.313106    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/142d0475-389b-4d22-9f20-45f676380a7d-kube-api-access-q7hq2" (OuterVolumeSpecName: "kube-api-access-q7hq2") pod "142d0475-389b-4d22-9f20-45f676380a7d" (UID: "142d0475-389b-4d22-9f20-45f676380a7d"). InnerVolumeSpecName "kube-api-access-q7hq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:20:03.313488 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.313470    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "142d0475-389b-4d22-9f20-45f676380a7d" (UID: "142d0475-389b-4d22-9f20-45f676380a7d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:03.313594 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.313570    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "142d0475-389b-4d22-9f20-45f676380a7d" (UID: "142d0475-389b-4d22-9f20-45f676380a7d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:03.412185 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.412136    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7hq2\" (UniqueName: \"kubernetes.io/projected/142d0475-389b-4d22-9f20-45f676380a7d-kube-api-access-q7hq2\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:03.412185 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.412179    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-serving-cert\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:03.412185 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.412189    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:03.412185 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.412198    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-oauth-serving-cert\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:03.412185 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.412207    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/142d0475-389b-4d22-9f20-45f676380a7d-console-config\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:03.412519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:03.412215    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/142d0475-389b-4d22-9f20-45f676380a7d-console-oauth-config\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:04.088105 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:04.088065    2583 generic.go:356] "Generic (PLEG): container finished" podID="142d0475-389b-4d22-9f20-45f676380a7d" containerID="a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a" exitCode=0
Sep 11 19:20:04.088105 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:04.088107    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8648b756-vqwlp"
Sep 11 19:20:04.088622 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:04.088135    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8648b756-vqwlp" event={"ID":"142d0475-389b-4d22-9f20-45f676380a7d","Type":"ContainerDied","Data":"a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a"}
Sep 11 19:20:04.088622 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:04.088188    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8648b756-vqwlp" event={"ID":"142d0475-389b-4d22-9f20-45f676380a7d","Type":"ContainerDied","Data":"6da9bab6a2cce76f41be79b70be8987440532fa80ade6a4afc451cda0a8ac088"}
Sep 11 19:20:04.088622 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:04.088216    2583 scope.go:117] "RemoveContainer" containerID="a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a"
Sep 11 19:20:04.098028 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:04.098011    2583 scope.go:117] "RemoveContainer" containerID="a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a"
Sep 11 19:20:04.098343 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:20:04.098320    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a\": container with ID starting with a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a not found: ID does not exist" containerID="a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a"
Sep 11 19:20:04.098424 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:04.098353    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a"} err="failed to get container status \"a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a\": rpc error: code = NotFound desc = could not find container \"a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a\": container with ID starting with a22e2734a28f3448ad65b799bd0415724b9b8eda89211ce005b56bb9faddbe9a not found: ID does not exist"
Sep 11 19:20:04.108949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:04.108913    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d8648b756-vqwlp"]
Sep 11 19:20:04.113765 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:04.113743    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d8648b756-vqwlp"]
Sep 11 19:20:04.549871 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:04.549829    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="142d0475-389b-4d22-9f20-45f676380a7d" path="/var/lib/kubelet/pods/142d0475-389b-4d22-9f20-45f676380a7d/volumes"
Sep 11 19:20:07.676609 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:07.676572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:20:07.777499 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:07.777462    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:20:15.011711 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.011661    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-96fc5559-zw5dg" podUID="5b169eaa-b951-4980-b7ff-0d4adeb818d6" containerName="console" containerID="cri-o://e43d0acfa017bcb8f5e4dbfa73cc804017e8c5fce57ac14bc9beb1b9ec1dd196" gracePeriod=15
Sep 11 19:20:15.137192 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.137162    2583 generic.go:356] "Generic (PLEG): container finished" podID="5b169eaa-b951-4980-b7ff-0d4adeb818d6" containerID="e43d0acfa017bcb8f5e4dbfa73cc804017e8c5fce57ac14bc9beb1b9ec1dd196" exitCode=0
Sep 11 19:20:15.137373 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.137214    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96fc5559-zw5dg" event={"ID":"5b169eaa-b951-4980-b7ff-0d4adeb818d6","Type":"ContainerDied","Data":"e43d0acfa017bcb8f5e4dbfa73cc804017e8c5fce57ac14bc9beb1b9ec1dd196"}
Sep 11 19:20:15.254134 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.254107    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:20:15.307359 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.307273    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-oauth-serving-cert\") pod \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") "
Sep 11 19:20:15.307359 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.307328    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-service-ca\") pod \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") "
Sep 11 19:20:15.307359 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.307349    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-trusted-ca-bundle\") pod \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") "
Sep 11 19:20:15.307635 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.307367    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcdnr\" (UniqueName: \"kubernetes.io/projected/5b169eaa-b951-4980-b7ff-0d4adeb818d6-kube-api-access-xcdnr\") pod \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") "
Sep 11 19:20:15.307635 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.307386    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-oauth-config\") pod \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") "
Sep 11 19:20:15.307635 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.307406    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-config\") pod \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") "
Sep 11 19:20:15.307635 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.307472    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-serving-cert\") pod \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\" (UID: \"5b169eaa-b951-4980-b7ff-0d4adeb818d6\") "
Sep 11 19:20:15.307862 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.307819    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5b169eaa-b951-4980-b7ff-0d4adeb818d6" (UID: "5b169eaa-b951-4980-b7ff-0d4adeb818d6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:15.307969 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.307945    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5b169eaa-b951-4980-b7ff-0d4adeb818d6" (UID: "5b169eaa-b951-4980-b7ff-0d4adeb818d6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:15.308027 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.307943    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-service-ca" (OuterVolumeSpecName: "service-ca") pod "5b169eaa-b951-4980-b7ff-0d4adeb818d6" (UID: "5b169eaa-b951-4980-b7ff-0d4adeb818d6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:15.308027 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.307970    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-config" (OuterVolumeSpecName: "console-config") pod "5b169eaa-b951-4980-b7ff-0d4adeb818d6" (UID: "5b169eaa-b951-4980-b7ff-0d4adeb818d6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:15.309853 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.309825    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5b169eaa-b951-4980-b7ff-0d4adeb818d6" (UID: "5b169eaa-b951-4980-b7ff-0d4adeb818d6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:15.309952 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.309852    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b169eaa-b951-4980-b7ff-0d4adeb818d6-kube-api-access-xcdnr" (OuterVolumeSpecName: "kube-api-access-xcdnr") pod "5b169eaa-b951-4980-b7ff-0d4adeb818d6" (UID: "5b169eaa-b951-4980-b7ff-0d4adeb818d6"). InnerVolumeSpecName "kube-api-access-xcdnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:20:15.309952 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.309901    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5b169eaa-b951-4980-b7ff-0d4adeb818d6" (UID: "5b169eaa-b951-4980-b7ff-0d4adeb818d6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:15.408914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.408874    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-trusted-ca-bundle\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:15.408914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.408905    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcdnr\" (UniqueName: \"kubernetes.io/projected/5b169eaa-b951-4980-b7ff-0d4adeb818d6-kube-api-access-xcdnr\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:15.408914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.408914    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-oauth-config\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:15.408914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.408922    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-config\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:15.409164 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.408931    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b169eaa-b951-4980-b7ff-0d4adeb818d6-console-serving-cert\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:15.409164 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.408939    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-oauth-serving-cert\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:15.409164 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:15.408947    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b169eaa-b951-4980-b7ff-0d4adeb818d6-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:16.142334 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:16.142287    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96fc5559-zw5dg" event={"ID":"5b169eaa-b951-4980-b7ff-0d4adeb818d6","Type":"ContainerDied","Data":"f929e9a22caf898383e4bd1707f91b1c5632fc9a17c1a7dbef23f6e0a39a7d8c"}
Sep 11 19:20:16.142334 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:16.142324    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96fc5559-zw5dg"
Sep 11 19:20:16.142859 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:16.142347    2583 scope.go:117] "RemoveContainer" containerID="e43d0acfa017bcb8f5e4dbfa73cc804017e8c5fce57ac14bc9beb1b9ec1dd196"
Sep 11 19:20:16.161773 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:16.161746    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-96fc5559-zw5dg"]
Sep 11 19:20:16.165642 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:16.165618    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-96fc5559-zw5dg"]
Sep 11 19:20:16.545069 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:16.545037    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b169eaa-b951-4980-b7ff-0d4adeb818d6" path="/var/lib/kubelet/pods/5b169eaa-b951-4980-b7ff-0d4adeb818d6/volumes"
Sep 11 19:20:17.106296 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:17.106264    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:20:17.110385 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:17.110341    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-69d66b8655-mnbcs"
Sep 11 19:20:39.116301 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:39.116267    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:39.136424 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:39.136388    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:39.244137 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:39.244107    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:51.792116 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:51.792075    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:20:51.992576 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:51.992534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:20:55.494789 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:55.494755    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:20:55.551022 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:55.550984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:20:57.186644 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.186606    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 19:20:57.187147 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.186880    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="prometheus" containerID="cri-o://0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce" gracePeriod=600
Sep 11 19:20:57.187147 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.186928    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="config-reloader" containerID="cri-o://36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8" gracePeriod=600
Sep 11 19:20:57.187147 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.186942    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129" gracePeriod=600
Sep 11 19:20:57.187147 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.186945    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="thanos-sidecar" containerID="cri-o://20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346" gracePeriod=600
Sep 11 19:20:57.187147 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.186960    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy-web" containerID="cri-o://df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b" gracePeriod=600
Sep 11 19:20:57.187147 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.186898    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy" containerID="cri-o://a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec" gracePeriod=600
Sep 11 19:20:57.429851 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.429825    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:57.440648 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.440580    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-db\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.440648 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.440611    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94m9m\" (UniqueName: \"kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-kube-api-access-94m9m\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.440648 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.440639    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-serving-certs-ca-bundle\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.440883 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.440668    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-metrics-client-certs\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.440883 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.440819    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-trusted-ca-bundle\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.440883 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.440871    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-tls\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441032 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.440906    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441032 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.440948    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441032 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.440981    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-kubelet-serving-ca-bundle\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441032 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441016    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-web-config\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441056    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-config\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441079    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-config-out\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441069    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:57.441210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441100    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-thanos-prometheus-http-client-file\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441140    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-grpc-tls\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441169    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-2\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441194    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-0\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441224    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-kube-rbac-proxy\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441399    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-tls-assets\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441478    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-metrics-client-ca\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441514    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-1\") pod \"385f5010-14f0-443d-b648-0354252fa18e\" (UID: \"385f5010-14f0-443d-b648-0354252fa18e\") "
Sep 11 19:20:57.441554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441528    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:57.441788 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441758    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.441788 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441777    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.442289 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.441998    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:20:57.442289 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.442125    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:57.442289 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.442196    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:57.442289 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.442208    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:57.443479 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.442768    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:57.443825 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.443799    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-kube-api-access-94m9m" (OuterVolumeSpecName: "kube-api-access-94m9m") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "kube-api-access-94m9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:20:57.444289 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.444224    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:20:57.444289 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.444233    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:57.445136 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.445108    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:57.445571 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.445543    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:20:57.445659 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.445608    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:57.446035 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.446009    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:57.446199 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.446176    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:57.446287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.446176    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-config-out" (OuterVolumeSpecName: "config-out") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:20:57.446444 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.446429    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:57.446746 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.446731    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:57.446806 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.446791    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-config" (OuterVolumeSpecName: "config") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:57.455757 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.455620    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-web-config" (OuterVolumeSpecName: "web-config") pod "385f5010-14f0-443d-b648-0354252fa18e" (UID: "385f5010-14f0-443d-b648-0354252fa18e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:20:57.542388 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542350    2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-tls-assets\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542388 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542382    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-metrics-client-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542388 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542391    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542400    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-db\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542409    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94m9m\" (UniqueName: \"kubernetes.io/projected/385f5010-14f0-443d-b648-0354252fa18e-kube-api-access-94m9m\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542419    2583 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-metrics-client-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542427    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542436    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542445    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542454    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542462    2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-web-config\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542469    2583 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-config\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542478    2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/385f5010-14f0-443d-b648-0354252fa18e-config-out\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542486    2583 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542494    2583 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-grpc-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542503    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542511    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/385f5010-14f0-443d-b648-0354252fa18e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:57.542621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:57.542519    2583 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/385f5010-14f0-443d-b648-0354252fa18e-secret-kube-rbac-proxy\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:20:58.298262 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298204    2583 generic.go:356] "Generic (PLEG): container finished" podID="385f5010-14f0-443d-b648-0354252fa18e" containerID="6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129" exitCode=0
Sep 11 19:20:58.298262 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298234    2583 generic.go:356] "Generic (PLEG): container finished" podID="385f5010-14f0-443d-b648-0354252fa18e" containerID="a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec" exitCode=0
Sep 11 19:20:58.298262 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298255    2583 generic.go:356] "Generic (PLEG): container finished" podID="385f5010-14f0-443d-b648-0354252fa18e" containerID="df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b" exitCode=0
Sep 11 19:20:58.298262 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298263    2583 generic.go:356] "Generic (PLEG): container finished" podID="385f5010-14f0-443d-b648-0354252fa18e" containerID="20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346" exitCode=0
Sep 11 19:20:58.298262 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298269    2583 generic.go:356] "Generic (PLEG): container finished" podID="385f5010-14f0-443d-b648-0354252fa18e" containerID="36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8" exitCode=0
Sep 11 19:20:58.298262 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298273    2583 generic.go:356] "Generic (PLEG): container finished" podID="385f5010-14f0-443d-b648-0354252fa18e" containerID="0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce" exitCode=0
Sep 11 19:20:58.298900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298331    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.298900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298333    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerDied","Data":"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"}
Sep 11 19:20:58.298900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298416    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerDied","Data":"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"}
Sep 11 19:20:58.298900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298461    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerDied","Data":"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"}
Sep 11 19:20:58.298900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298502    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerDied","Data":"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"}
Sep 11 19:20:58.298900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298553    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerDied","Data":"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"}
Sep 11 19:20:58.298900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298499    2583 scope.go:117] "RemoveContainer" containerID="6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"
Sep 11 19:20:58.298900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298593    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerDied","Data":"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"}
Sep 11 19:20:58.298900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.298712    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"385f5010-14f0-443d-b648-0354252fa18e","Type":"ContainerDied","Data":"7d23e81be15234e87c10a4f519180bc43139741bf838c8df7abaa27f7fe8c80f"}
Sep 11 19:20:58.306538 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.306518    2583 scope.go:117] "RemoveContainer" containerID="a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"
Sep 11 19:20:58.313742 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.313724    2583 scope.go:117] "RemoveContainer" containerID="df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"
Sep 11 19:20:58.320464 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.320445    2583 scope.go:117] "RemoveContainer" containerID="20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"
Sep 11 19:20:58.323061 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.323040    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 19:20:58.329405 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.329314    2583 scope.go:117] "RemoveContainer" containerID="36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"
Sep 11 19:20:58.331551 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.330367    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 19:20:58.338771 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.338747    2583 scope.go:117] "RemoveContainer" containerID="0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"
Sep 11 19:20:58.345981 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.345967    2583 scope.go:117] "RemoveContainer" containerID="222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"
Sep 11 19:20:58.352725 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.352709    2583 scope.go:117] "RemoveContainer" containerID="6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"
Sep 11 19:20:58.352981 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:20:58.352949    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": container with ID starting with 6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129 not found: ID does not exist" containerID="6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"
Sep 11 19:20:58.353024 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.352990    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"} err="failed to get container status \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": rpc error: code = NotFound desc = could not find container \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": container with ID starting with 6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129 not found: ID does not exist"
Sep 11 19:20:58.353024 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.353009    2583 scope.go:117] "RemoveContainer" containerID="a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"
Sep 11 19:20:58.353280 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:20:58.353262    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": container with ID starting with a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec not found: ID does not exist" containerID="a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"
Sep 11 19:20:58.353326 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.353287    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"} err="failed to get container status \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": rpc error: code = NotFound desc = could not find container \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": container with ID starting with a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec not found: ID does not exist"
Sep 11 19:20:58.353326 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.353304    2583 scope.go:117] "RemoveContainer" containerID="df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"
Sep 11 19:20:58.353565 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:20:58.353548    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": container with ID starting with df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b not found: ID does not exist" containerID="df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"
Sep 11 19:20:58.353621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.353570    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"} err="failed to get container status \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": rpc error: code = NotFound desc = could not find container \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": container with ID starting with df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b not found: ID does not exist"
Sep 11 19:20:58.353621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.353585    2583 scope.go:117] "RemoveContainer" containerID="20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"
Sep 11 19:20:58.353848 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:20:58.353834    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": container with ID starting with 20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346 not found: ID does not exist" containerID="20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"
Sep 11 19:20:58.353893 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.353850    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"} err="failed to get container status \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": rpc error: code = NotFound desc = could not find container \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": container with ID starting with 20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346 not found: ID does not exist"
Sep 11 19:20:58.353893 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.353862    2583 scope.go:117] "RemoveContainer" containerID="36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"
Sep 11 19:20:58.354081 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:20:58.354065    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": container with ID starting with 36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8 not found: ID does not exist" containerID="36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"
Sep 11 19:20:58.354121 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.354085    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"} err="failed to get container status \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": rpc error: code = NotFound desc = could not find container \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": container with ID starting with 36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8 not found: ID does not exist"
Sep 11 19:20:58.354121 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.354099    2583 scope.go:117] "RemoveContainer" containerID="0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"
Sep 11 19:20:58.354403 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:20:58.354384    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": container with ID starting with 0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce not found: ID does not exist" containerID="0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"
Sep 11 19:20:58.354486 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.354411    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"} err="failed to get container status \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": rpc error: code = NotFound desc = could not find container \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": container with ID starting with 0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce not found: ID does not exist"
Sep 11 19:20:58.354486 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.354433    2583 scope.go:117] "RemoveContainer" containerID="222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"
Sep 11 19:20:58.354705 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:20:58.354691    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": container with ID starting with 222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018 not found: ID does not exist" containerID="222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"
Sep 11 19:20:58.354769 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.354711    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"} err="failed to get container status \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": rpc error: code = NotFound desc = could not find container \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": container with ID starting with 222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018 not found: ID does not exist"
Sep 11 19:20:58.354769 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.354729    2583 scope.go:117] "RemoveContainer" containerID="6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"
Sep 11 19:20:58.354970 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.354952    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"} err="failed to get container status \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": rpc error: code = NotFound desc = could not find container \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": container with ID starting with 6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129 not found: ID does not exist"
Sep 11 19:20:58.355036 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.354972    2583 scope.go:117] "RemoveContainer" containerID="a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"
Sep 11 19:20:58.355221 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.355200    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"} err="failed to get container status \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": rpc error: code = NotFound desc = could not find container \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": container with ID starting with a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec not found: ID does not exist"
Sep 11 19:20:58.355288 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.355222    2583 scope.go:117] "RemoveContainer" containerID="df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"
Sep 11 19:20:58.355491 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.355475    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"} err="failed to get container status \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": rpc error: code = NotFound desc = could not find container \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": container with ID starting with df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b not found: ID does not exist"
Sep 11 19:20:58.355553 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.355493    2583 scope.go:117] "RemoveContainer" containerID="20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"
Sep 11 19:20:58.355808 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.355781    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"} err="failed to get container status \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": rpc error: code = NotFound desc = could not find container \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": container with ID starting with 20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346 not found: ID does not exist"
Sep 11 19:20:58.355808 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.355807    2583 scope.go:117] "RemoveContainer" containerID="36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"
Sep 11 19:20:58.356210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.356156    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"} err="failed to get container status \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": rpc error: code = NotFound desc = could not find container \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": container with ID starting with 36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8 not found: ID does not exist"
Sep 11 19:20:58.356210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.356183    2583 scope.go:117] "RemoveContainer" containerID="0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"
Sep 11 19:20:58.356644 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.356616    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"} err="failed to get container status \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": rpc error: code = NotFound desc = could not find container \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": container with ID starting with 0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce not found: ID does not exist"
Sep 11 19:20:58.356644 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.356641    2583 scope.go:117] "RemoveContainer" containerID="222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"
Sep 11 19:20:58.356924 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.356905    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"} err="failed to get container status \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": rpc error: code = NotFound desc = could not find container \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": container with ID starting with 222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018 not found: ID does not exist"
Sep 11 19:20:58.357010 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.356927    2583 scope.go:117] "RemoveContainer" containerID="6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"
Sep 11 19:20:58.357213 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357194    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"} err="failed to get container status \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": rpc error: code = NotFound desc = could not find container \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": container with ID starting with 6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129 not found: ID does not exist"
Sep 11 19:20:58.357284 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357213    2583 scope.go:117] "RemoveContainer" containerID="a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"
Sep 11 19:20:58.357487 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357460    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"} err="failed to get container status \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": rpc error: code = NotFound desc = could not find container \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": container with ID starting with a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec not found: ID does not exist"
Sep 11 19:20:58.357564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357488    2583 scope.go:117] "RemoveContainer" containerID="df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"
Sep 11 19:20:58.357564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357532    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 19:20:58.357788 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357765    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"} err="failed to get container status \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": rpc error: code = NotFound desc = could not find container \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": container with ID starting with df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b not found: ID does not exist"
Sep 11 19:20:58.357842 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357791    2583 scope.go:117] "RemoveContainer" containerID="20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"
Sep 11 19:20:58.357883 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357860    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy-web"
Sep 11 19:20:58.357883 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357874    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy-web"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357885    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="prometheus"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357891    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="prometheus"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357901    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="thanos-sidecar"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357907    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="thanos-sidecar"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357917    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy-thanos"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357922    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy-thanos"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357929    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b169eaa-b951-4980-b7ff-0d4adeb818d6" containerName="console"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357934    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b169eaa-b951-4980-b7ff-0d4adeb818d6" containerName="console"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357942    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="init-config-reloader"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357949    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="init-config-reloader"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357958    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357965    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357974    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142d0475-389b-4d22-9f20-45f676380a7d" containerName="console"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357983    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="142d0475-389b-4d22-9f20-45f676380a7d" containerName="console"
Sep 11 19:20:58.357982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357992    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="config-reloader"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.357999    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="config-reloader"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358050    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="thanos-sidecar"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358057    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358064    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy-web"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358070    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="prometheus"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358077    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="config-reloader"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358081    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="385f5010-14f0-443d-b648-0354252fa18e" containerName="kube-rbac-proxy-thanos"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358086    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="142d0475-389b-4d22-9f20-45f676380a7d" containerName="console"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358080    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"} err="failed to get container status \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": rpc error: code = NotFound desc = could not find container \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": container with ID starting with 20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346 not found: ID does not exist"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358107    2583 scope.go:117] "RemoveContainer" containerID="36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358090    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b169eaa-b951-4980-b7ff-0d4adeb818d6" containerName="console"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358379    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"} err="failed to get container status \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": rpc error: code = NotFound desc = could not find container \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": container with ID starting with 36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8 not found: ID does not exist"
Sep 11 19:20:58.358565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358394    2583 scope.go:117] "RemoveContainer" containerID="0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"
Sep 11 19:20:58.359070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358631    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"} err="failed to get container status \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": rpc error: code = NotFound desc = could not find container \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": container with ID starting with 0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce not found: ID does not exist"
Sep 11 19:20:58.359070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358645    2583 scope.go:117] "RemoveContainer" containerID="222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"
Sep 11 19:20:58.359070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358898    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"} err="failed to get container status \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": rpc error: code = NotFound desc = could not find container \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": container with ID starting with 222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018 not found: ID does not exist"
Sep 11 19:20:58.359070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.358940    2583 scope.go:117] "RemoveContainer" containerID="6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"
Sep 11 19:20:58.359297 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.359281    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"} err="failed to get container status \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": rpc error: code = NotFound desc = could not find container \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": container with ID starting with 6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129 not found: ID does not exist"
Sep 11 19:20:58.359360 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.359297    2583 scope.go:117] "RemoveContainer" containerID="a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"
Sep 11 19:20:58.359589 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.359568    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"} err="failed to get container status \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": rpc error: code = NotFound desc = could not find container \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": container with ID starting with a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec not found: ID does not exist"
Sep 11 19:20:58.359637 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.359590    2583 scope.go:117] "RemoveContainer" containerID="df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"
Sep 11 19:20:58.359843 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.359827    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"} err="failed to get container status \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": rpc error: code = NotFound desc = could not find container \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": container with ID starting with df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b not found: ID does not exist"
Sep 11 19:20:58.359898 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.359844    2583 scope.go:117] "RemoveContainer" containerID="20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"
Sep 11 19:20:58.360016 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.360002    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"} err="failed to get container status \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": rpc error: code = NotFound desc = could not find container \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": container with ID starting with 20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346 not found: ID does not exist"
Sep 11 19:20:58.360069 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.360017    2583 scope.go:117] "RemoveContainer" containerID="36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"
Sep 11 19:20:58.360233 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.360215    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"} err="failed to get container status \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": rpc error: code = NotFound desc = could not find container \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": container with ID starting with 36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8 not found: ID does not exist"
Sep 11 19:20:58.360233 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.360234    2583 scope.go:117] "RemoveContainer" containerID="0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"
Sep 11 19:20:58.360508 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.360489    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"} err="failed to get container status \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": rpc error: code = NotFound desc = could not find container \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": container with ID starting with 0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce not found: ID does not exist"
Sep 11 19:20:58.360561 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.360509    2583 scope.go:117] "RemoveContainer" containerID="222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"
Sep 11 19:20:58.360762 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.360747    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"} err="failed to get container status \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": rpc error: code = NotFound desc = could not find container \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": container with ID starting with 222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018 not found: ID does not exist"
Sep 11 19:20:58.360822 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.360764    2583 scope.go:117] "RemoveContainer" containerID="6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"
Sep 11 19:20:58.361002 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.360985    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"} err="failed to get container status \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": rpc error: code = NotFound desc = could not find container \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": container with ID starting with 6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129 not found: ID does not exist"
Sep 11 19:20:58.361075 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.361005    2583 scope.go:117] "RemoveContainer" containerID="a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"
Sep 11 19:20:58.361224 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.361206    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"} err="failed to get container status \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": rpc error: code = NotFound desc = could not find container \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": container with ID starting with a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec not found: ID does not exist"
Sep 11 19:20:58.361296 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.361226    2583 scope.go:117] "RemoveContainer" containerID="df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"
Sep 11 19:20:58.361461 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.361443    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"} err="failed to get container status \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": rpc error: code = NotFound desc = could not find container \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": container with ID starting with df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b not found: ID does not exist"
Sep 11 19:20:58.361505 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.361462    2583 scope.go:117] "RemoveContainer" containerID="20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"
Sep 11 19:20:58.361687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.361661    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"} err="failed to get container status \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": rpc error: code = NotFound desc = could not find container \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": container with ID starting with 20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346 not found: ID does not exist"
Sep 11 19:20:58.361748 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.361688    2583 scope.go:117] "RemoveContainer" containerID="36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"
Sep 11 19:20:58.361932 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.361915    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"} err="failed to get container status \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": rpc error: code = NotFound desc = could not find container \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": container with ID starting with 36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8 not found: ID does not exist"
Sep 11 19:20:58.361932 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.361931    2583 scope.go:117] "RemoveContainer" containerID="0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"
Sep 11 19:20:58.362124 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.362104    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"} err="failed to get container status \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": rpc error: code = NotFound desc = could not find container \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": container with ID starting with 0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce not found: ID does not exist"
Sep 11 19:20:58.362190 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.362126    2583 scope.go:117] "RemoveContainer" containerID="222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"
Sep 11 19:20:58.362351 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.362327    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"} err="failed to get container status \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": rpc error: code = NotFound desc = could not find container \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": container with ID starting with 222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018 not found: ID does not exist"
Sep 11 19:20:58.362395 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.362351    2583 scope.go:117] "RemoveContainer" containerID="6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"
Sep 11 19:20:58.362567 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.362549    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129"} err="failed to get container status \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": rpc error: code = NotFound desc = could not find container \"6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129\": container with ID starting with 6baf62734132aa3b5808e257a1b5bb361cc3c1c9504d85fc40035b374bf0d129 not found: ID does not exist"
Sep 11 19:20:58.362634 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.362568    2583 scope.go:117] "RemoveContainer" containerID="a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"
Sep 11 19:20:58.362825 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.362804    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec"} err="failed to get container status \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": rpc error: code = NotFound desc = could not find container \"a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec\": container with ID starting with a521b4bd92cb1e9ebf38a217e8149bb99c095f57ccb4f0c710bbca9101867aec not found: ID does not exist"
Sep 11 19:20:58.362825 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.362825    2583 scope.go:117] "RemoveContainer" containerID="df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"
Sep 11 19:20:58.363020 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.363001    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b"} err="failed to get container status \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": rpc error: code = NotFound desc = could not find container \"df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b\": container with ID starting with df3a9863da004ac7ec5789cd9c43d75d8b474d471d86ec03b3b549096898240b not found: ID does not exist"
Sep 11 19:20:58.363087 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.363022    2583 scope.go:117] "RemoveContainer" containerID="20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"
Sep 11 19:20:58.363300 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.363280    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346"} err="failed to get container status \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": rpc error: code = NotFound desc = could not find container \"20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346\": container with ID starting with 20571c6b387ba88f90266c771be0a152062675d7ded24d3cc212e2060245d346 not found: ID does not exist"
Sep 11 19:20:58.363381 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.363304    2583 scope.go:117] "RemoveContainer" containerID="36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"
Sep 11 19:20:58.363558 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.363540    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8"} err="failed to get container status \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": rpc error: code = NotFound desc = could not find container \"36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8\": container with ID starting with 36c6ab8378411bf6f6f57cb624f791d1e0aecc4b6f1e1c76c197da7661e5c6a8 not found: ID does not exist"
Sep 11 19:20:58.363607 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.363559    2583 scope.go:117] "RemoveContainer" containerID="0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"
Sep 11 19:20:58.363762 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.363741    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce"} err="failed to get container status \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": rpc error: code = NotFound desc = could not find container \"0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce\": container with ID starting with 0797eb845097e7f640663c041dcaefd518f5f9dd5bccf75619ddfda1562185ce not found: ID does not exist"
Sep 11 19:20:58.363835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.363763    2583 scope.go:117] "RemoveContainer" containerID="222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"
Sep 11 19:20:58.363835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.363811    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.364157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.364043    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018"} err="failed to get container status \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": rpc error: code = NotFound desc = could not find container \"222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018\": container with ID starting with 222c7879e158f0a0c3b4eaccb05894ad05733afef5eadbc42f1500eaf47e0018 not found: ID does not exist"
Sep 11 19:20:58.366396 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366173    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 19:20:58.366396 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366219    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 19:20:58.366396 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366221    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 19:20:58.366396 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366234    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 19:20:58.366396 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366292    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9p662s6vcma48\""
Sep 11 19:20:58.366396 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366172    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 19:20:58.366747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366420    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7v5pw\""
Sep 11 19:20:58.366747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366447    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 19:20:58.366747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366487    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 19:20:58.366747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366533    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 19:20:58.366747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366543    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 19:20:58.366747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366553    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 19:20:58.366747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366560    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 19:20:58.366747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.366726    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 19:20:58.369386 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.369365    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 19:20:58.371376 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.371333    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 19:20:58.378587 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.378567    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 19:20:58.449454 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449411    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2lw4\" (UniqueName: \"kubernetes.io/projected/5d3431ea-1dd1-4977-876f-7a7d27ae2867-kube-api-access-t2lw4\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.449656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449462    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d3431ea-1dd1-4977-876f-7a7d27ae2867-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.449656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449488    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.449656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449516    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.449656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449555    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.449656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449574    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-web-config\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.449656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449599    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.449656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449622    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-config\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449709    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449739    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449761    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449817    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d3431ea-1dd1-4977-876f-7a7d27ae2867-config-out\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449854    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449887    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449904    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449947    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449963    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.450012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.449985    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.545637 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.545605    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="385f5010-14f0-443d-b648-0354252fa18e" path="/var/lib/kubelet/pods/385f5010-14f0-443d-b648-0354252fa18e/volumes"
Sep 11 19:20:58.550425 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550406    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d3431ea-1dd1-4977-876f-7a7d27ae2867-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550506 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550435    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550506 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550452    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550506 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550506 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550484    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-web-config\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550681 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550619    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550681 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550655    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-config\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550753 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550693    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550753 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550725    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550759    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550782    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550808    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d3431ea-1dd1-4977-876f-7a7d27ae2867-config-out\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.550835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550832    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.551042 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550859    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.551042 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.551042 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550908    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.551042 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.551042 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.550959    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.551042 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.551000    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.551042 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.551039    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2lw4\" (UniqueName: \"kubernetes.io/projected/5d3431ea-1dd1-4977-876f-7a7d27ae2867-kube-api-access-t2lw4\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.551553 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.551132    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.551553 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.551215    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.552881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.552020    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.552881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.552122    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.552881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.552591    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.554416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.554000    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d3431ea-1dd1-4977-876f-7a7d27ae2867-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.554416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.554016    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.554416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.554003    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-config\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.554416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.554079    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-web-config\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.554416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.554412    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.554724 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.554662    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.555117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.555091    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.555295 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.555269    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5d3431ea-1dd1-4977-876f-7a7d27ae2867-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.555606 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.555587    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.556194 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.556169    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.556531 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.556511    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.556933 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.556913    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d3431ea-1dd1-4977-876f-7a7d27ae2867-config-out\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.557221 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.557206    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.557307 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.557290    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d3431ea-1dd1-4977-876f-7a7d27ae2867-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.559432 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.559412    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2lw4\" (UniqueName: \"kubernetes.io/projected/5d3431ea-1dd1-4977-876f-7a7d27ae2867-kube-api-access-t2lw4\") pod \"prometheus-k8s-0\" (UID: \"5d3431ea-1dd1-4977-876f-7a7d27ae2867\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.673695 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.673645    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:20:58.954512 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:58.954462    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 19:20:58.956570 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:20:58.956544    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3431ea_1dd1_4977_876f_7a7d27ae2867.slice/crio-c511778fc59ee70186ef2cc368f70dee4642d5aa99697bf7ad3fa8ccb9843904 WatchSource:0}: Error finding container c511778fc59ee70186ef2cc368f70dee4642d5aa99697bf7ad3fa8ccb9843904: Status 404 returned error can't find the container with id c511778fc59ee70186ef2cc368f70dee4642d5aa99697bf7ad3fa8ccb9843904
Sep 11 19:20:59.304856 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:59.304760    2583 generic.go:356] "Generic (PLEG): container finished" podID="5d3431ea-1dd1-4977-876f-7a7d27ae2867" containerID="b3ec5f09d1899e292b3a123bf8c5cc4d14d60f3ad6c109d23e080320db2ab97e" exitCode=0
Sep 11 19:20:59.304856 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:59.304827    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d3431ea-1dd1-4977-876f-7a7d27ae2867","Type":"ContainerDied","Data":"b3ec5f09d1899e292b3a123bf8c5cc4d14d60f3ad6c109d23e080320db2ab97e"}
Sep 11 19:20:59.305292 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:20:59.304874    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d3431ea-1dd1-4977-876f-7a7d27ae2867","Type":"ContainerStarted","Data":"c511778fc59ee70186ef2cc368f70dee4642d5aa99697bf7ad3fa8ccb9843904"}
Sep 11 19:21:00.312630 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:00.312597    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d3431ea-1dd1-4977-876f-7a7d27ae2867","Type":"ContainerStarted","Data":"3b05fb2ab2822fbd18d9adb667b6605a6a83ba6d91adf181c2d4653020dad1cb"}
Sep 11 19:21:00.313061 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:00.312691    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d3431ea-1dd1-4977-876f-7a7d27ae2867","Type":"ContainerStarted","Data":"b22762607a89c2bb6765975f88a8d927c08e816ef5c6fbdca942f112be87b652"}
Sep 11 19:21:00.313061 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:00.312720    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d3431ea-1dd1-4977-876f-7a7d27ae2867","Type":"ContainerStarted","Data":"22b2fe6c3b5a9063a389f24594d71c83606f4681f8a1ec9741e48cff1e715f15"}
Sep 11 19:21:00.313061 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:00.312759    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d3431ea-1dd1-4977-876f-7a7d27ae2867","Type":"ContainerStarted","Data":"d187a6307b1607b05a9d0dc6509246a5e02dbc6fc5688f7f544e98da4c026a2d"}
Sep 11 19:21:00.313061 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:00.312788    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d3431ea-1dd1-4977-876f-7a7d27ae2867","Type":"ContainerStarted","Data":"db034d3a67f31e74867498799e550140c27f4d31c02787ef2c3fdc215cbb12b8"}
Sep 11 19:21:00.313061 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:00.312819    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d3431ea-1dd1-4977-876f-7a7d27ae2867","Type":"ContainerStarted","Data":"edc81e10efe7cd490286ef2cab9ed745e8cfd17fbc4e793d923e63be68efef3a"}
Sep 11 19:21:00.347862 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:00.347809    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.328720532 podStartE2EDuration="2.347791272s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.07074ms" isStatefulPod=false podCreationTimestamp="2026-09-11 19:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:21:00.344494033 +0000 UTC m=+164.273118943" watchObservedRunningTime="2026-09-11 19:21:00.347791272 +0000 UTC m=+164.276416169"
Sep 11 19:21:00.414451 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:00.414407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:21:00.512761 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:00.512731    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:21:03.674719 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:03.674671    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:21:11.415529 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:11.415447    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:21:11.516885 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:11.516851    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:21:11.573523 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:11.573497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:21:29.745073 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:29.745036    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:21:58.674765 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:58.674727    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:21:58.690856 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:58.690826    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:21:59.550938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:21:59.550910    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 19:22:07.085975 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:07.085939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:07.147346 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:07.147314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:08.468189 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:08.468143    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:09.057550 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:09.057515    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:09.118632 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:09.118601    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:09.190636 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:09.190608    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:09.240259 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:09.240218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:10.009603 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:10.009570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:10.061621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:10.061588    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:19.051517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:19.051482    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:19.113118 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:19.113085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:19.656502 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:19.656469    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:19.756868 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:19.756828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:19.815601 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:19.815570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:22.684472 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:22.684415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:22.785432 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:22.785393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:40.360904 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:40.360814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:40.425096 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:40.425060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:58.470754 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:58.470722    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:22:58.565085 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:22:58.565055    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:16.463794 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:16.463771    2583 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 19:23:25.406706 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:25.406675    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:26.530870 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:26.530837    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:29.235322 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:29.235290    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:29.336440 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:29.336403    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:30.929150 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:30.929115    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:31.047457 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:31.047422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:31.236945 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:31.236864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:31.349091 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:31.349058    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:32.049316 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.049278    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jdj7w"]
Sep 11 19:23:32.052393 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.052371    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w"
Sep 11 19:23:32.054945 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.054925    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 19:23:32.055737 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.055714    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 19:23:32.055789 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.055764    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-q9d2f\""
Sep 11 19:23:32.062157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.062135    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jdj7w"]
Sep 11 19:23:32.159179 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.159141    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58ea49af-b939-4456-bd5d-e0c2c4771a76-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jdj7w\" (UID: \"58ea49af-b939-4456-bd5d-e0c2c4771a76\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w"
Sep 11 19:23:32.159420 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.159214    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkk4p\" (UniqueName: \"kubernetes.io/projected/58ea49af-b939-4456-bd5d-e0c2c4771a76-kube-api-access-rkk4p\") pod \"cert-manager-webhook-6486ffd997-jdj7w\" (UID: \"58ea49af-b939-4456-bd5d-e0c2c4771a76\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w"
Sep 11 19:23:32.260454 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.260409    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58ea49af-b939-4456-bd5d-e0c2c4771a76-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jdj7w\" (UID: \"58ea49af-b939-4456-bd5d-e0c2c4771a76\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w"
Sep 11 19:23:32.260631 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.260483    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkk4p\" (UniqueName: \"kubernetes.io/projected/58ea49af-b939-4456-bd5d-e0c2c4771a76-kube-api-access-rkk4p\") pod \"cert-manager-webhook-6486ffd997-jdj7w\" (UID: \"58ea49af-b939-4456-bd5d-e0c2c4771a76\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w"
Sep 11 19:23:32.268728 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.268705    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58ea49af-b939-4456-bd5d-e0c2c4771a76-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jdj7w\" (UID: \"58ea49af-b939-4456-bd5d-e0c2c4771a76\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w"
Sep 11 19:23:32.268849 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.268766    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkk4p\" (UniqueName: \"kubernetes.io/projected/58ea49af-b939-4456-bd5d-e0c2c4771a76-kube-api-access-rkk4p\") pod \"cert-manager-webhook-6486ffd997-jdj7w\" (UID: \"58ea49af-b939-4456-bd5d-e0c2c4771a76\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w"
Sep 11 19:23:32.367656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.367628    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w"
Sep 11 19:23:32.639505 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.639424    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jdj7w"]
Sep 11 19:23:32.643709 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:23:32.643682    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58ea49af_b939_4456_bd5d_e0c2c4771a76.slice/crio-7da2b2f87ec6ba6cd28590b23f18acf3872a7e2249fea2cb129e6dcdb8440ca0 WatchSource:0}: Error finding container 7da2b2f87ec6ba6cd28590b23f18acf3872a7e2249fea2cb129e6dcdb8440ca0: Status 404 returned error can't find the container with id 7da2b2f87ec6ba6cd28590b23f18acf3872a7e2249fea2cb129e6dcdb8440ca0
Sep 11 19:23:32.645555 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.645537    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 19:23:32.864326 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:32.864288    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w" event={"ID":"58ea49af-b939-4456-bd5d-e0c2c4771a76","Type":"ContainerStarted","Data":"7da2b2f87ec6ba6cd28590b23f18acf3872a7e2249fea2cb129e6dcdb8440ca0"}
Sep 11 19:23:33.369777 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:33.369744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:33.534613 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:33.534558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:36.033164 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:36.033137    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:36.165394 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:36.165357    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:36.883311 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:36.883276    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w" event={"ID":"58ea49af-b939-4456-bd5d-e0c2c4771a76","Type":"ContainerStarted","Data":"025476e2c62fccf23f7bddaa312710a9dcffa13b697485e7d13750d8ee9a7ce4"}
Sep 11 19:23:36.883492 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:36.883400    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w"
Sep 11 19:23:36.915579 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:36.915527    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w" podStartSLOduration=1.576442019 podStartE2EDuration="4.915510581s" totalImagesPullingTime="3.339068562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:23:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:23:36.914588982 +0000 UTC m=+320.843213880" watchObservedRunningTime="2026-09-11 19:23:36.915510581 +0000 UTC m=+320.844135480"
Sep 11 19:23:41.082326 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:41.082291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:42.215485 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:42.215445    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:42.316844 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:42.316811    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:42.888908 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:42.888882    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jdj7w"
Sep 11 19:23:44.255752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.255715    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf"]
Sep 11 19:23:44.258949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.258925    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf"
Sep 11 19:23:44.261668 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.261647    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 19:23:44.262340 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.262323    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 19:23:44.262431 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.262324    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-tmkc8\""
Sep 11 19:23:44.268604 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.268583    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf"]
Sep 11 19:23:44.362681 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.362629    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs2gr\" (UniqueName: \"kubernetes.io/projected/ded261e5-0a1d-4764-8df4-33bf5b177438-kube-api-access-zs2gr\") pod \"openshift-lws-operator-fd8ccff4c-r72jf\" (UID: \"ded261e5-0a1d-4764-8df4-33bf5b177438\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf"
Sep 11 19:23:44.362681 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.362693    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ded261e5-0a1d-4764-8df4-33bf5b177438-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r72jf\" (UID: \"ded261e5-0a1d-4764-8df4-33bf5b177438\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf"
Sep 11 19:23:44.463482 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.463432    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs2gr\" (UniqueName: \"kubernetes.io/projected/ded261e5-0a1d-4764-8df4-33bf5b177438-kube-api-access-zs2gr\") pod \"openshift-lws-operator-fd8ccff4c-r72jf\" (UID: \"ded261e5-0a1d-4764-8df4-33bf5b177438\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf"
Sep 11 19:23:44.463482 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.463493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ded261e5-0a1d-4764-8df4-33bf5b177438-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r72jf\" (UID: \"ded261e5-0a1d-4764-8df4-33bf5b177438\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf"
Sep 11 19:23:44.463813 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.463798    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ded261e5-0a1d-4764-8df4-33bf5b177438-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r72jf\" (UID: \"ded261e5-0a1d-4764-8df4-33bf5b177438\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf"
Sep 11 19:23:44.471589 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.471562    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs2gr\" (UniqueName: \"kubernetes.io/projected/ded261e5-0a1d-4764-8df4-33bf5b177438-kube-api-access-zs2gr\") pod \"openshift-lws-operator-fd8ccff4c-r72jf\" (UID: \"ded261e5-0a1d-4764-8df4-33bf5b177438\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf"
Sep 11 19:23:44.567638 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.567549    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf"
Sep 11 19:23:44.841997 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.841967    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf"]
Sep 11 19:23:44.844724 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:23:44.844694    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podded261e5_0a1d_4764_8df4_33bf5b177438.slice/crio-3fe1d552ae93492007d984eeb945b24eee93f3f8fb6b40b8dfcb61fc91b2001b WatchSource:0}: Error finding container 3fe1d552ae93492007d984eeb945b24eee93f3f8fb6b40b8dfcb61fc91b2001b: Status 404 returned error can't find the container with id 3fe1d552ae93492007d984eeb945b24eee93f3f8fb6b40b8dfcb61fc91b2001b
Sep 11 19:23:44.912903 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:44.912869    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf" event={"ID":"ded261e5-0a1d-4764-8df4-33bf5b177438","Type":"ContainerStarted","Data":"3fe1d552ae93492007d984eeb945b24eee93f3f8fb6b40b8dfcb61fc91b2001b"}
Sep 11 19:23:47.926903 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:47.926871    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf" event={"ID":"ded261e5-0a1d-4764-8df4-33bf5b177438","Type":"ContainerStarted","Data":"a068b98e685908d15cd5e8426ec41a9013fdd1f8b8b40eb9252e1a3745aa0282"}
Sep 11 19:23:47.943449 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:47.943399    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r72jf" podStartSLOduration=1.218933297 podStartE2EDuration="3.94338186s" totalImagesPullingTime="2.724448563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:23:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:23:47.941854464 +0000 UTC m=+331.870479367" watchObservedRunningTime="2026-09-11 19:23:47.94338186 +0000 UTC m=+331.872006758"
Sep 11 19:23:48.627497 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:48.627462    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:48.728339 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:48.728300    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:23:50.624050 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:23:50.624015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:04.835563 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:04.835531    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:04.935108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:04.935073    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:05.441076 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:05.441040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:05.804191 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:05.804108    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:05.913752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:05.913715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:05.953713 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:05.953677    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"]
Sep 11 19:24:05.960257 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:05.960220    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:05.962705 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:05.962683    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 19:24:05.962826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:05.962785    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 19:24:05.962874 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:05.962823    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 19:24:05.963086 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:05.963071    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 19:24:05.963427 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:05.963412    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xtpql\""
Sep 11 19:24:05.977796 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:05.977772    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"]
Sep 11 19:24:06.014198 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.014164    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/603632b5-75d8-4672-8126-88b4b338c3b0-tmp\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.014414 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.014230    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/603632b5-75d8-4672-8126-88b4b338c3b0-manifests\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.014414 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.014308    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqkb7\" (UniqueName: \"kubernetes.io/projected/603632b5-75d8-4672-8126-88b4b338c3b0-kube-api-access-cqkb7\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.014414 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.014368    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/603632b5-75d8-4672-8126-88b4b338c3b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.014414 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.014399    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/603632b5-75d8-4672-8126-88b4b338c3b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.074942 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.074857    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:06.115502 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.115467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/603632b5-75d8-4672-8126-88b4b338c3b0-manifests\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.115687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.115504    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkb7\" (UniqueName: \"kubernetes.io/projected/603632b5-75d8-4672-8126-88b4b338c3b0-kube-api-access-cqkb7\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.115687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.115543    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/603632b5-75d8-4672-8126-88b4b338c3b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.115687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.115586    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/603632b5-75d8-4672-8126-88b4b338c3b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.115687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.115631    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/603632b5-75d8-4672-8126-88b4b338c3b0-tmp\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.115903 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.115857    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/603632b5-75d8-4672-8126-88b4b338c3b0-manifests\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.118070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.118041    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/603632b5-75d8-4672-8126-88b4b338c3b0-tmp\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.118201 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.118091    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/603632b5-75d8-4672-8126-88b4b338c3b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.118288 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.118253    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/603632b5-75d8-4672-8126-88b4b338c3b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.123609 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.123588    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqkb7\" (UniqueName: \"kubernetes.io/projected/603632b5-75d8-4672-8126-88b4b338c3b0-kube-api-access-cqkb7\") pod \"opendatahub-operator-controller-manager-6474799d6-hj8vr\" (UID: \"603632b5-75d8-4672-8126-88b4b338c3b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.195397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.195359    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:06.270654 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.270622    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:06.275868 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.275845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:06.376385 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.376358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:06.476858 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.476828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:06.552852 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.552819    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"]
Sep 11 19:24:06.554679 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.554641    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:06.558449 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:24:06.558423    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603632b5_75d8_4672_8126_88b4b338c3b0.slice/crio-60af24c7cd2686a03c65ccb311c9cc7ce92895983dc0bec49e5ac394f1e6c1af WatchSource:0}: Error finding container 60af24c7cd2686a03c65ccb311c9cc7ce92895983dc0bec49e5ac394f1e6c1af: Status 404 returned error can't find the container with id 60af24c7cd2686a03c65ccb311c9cc7ce92895983dc0bec49e5ac394f1e6c1af
Sep 11 19:24:06.998358 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:06.998319    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr" event={"ID":"603632b5-75d8-4672-8126-88b4b338c3b0","Type":"ContainerStarted","Data":"60af24c7cd2686a03c65ccb311c9cc7ce92895983dc0bec49e5ac394f1e6c1af"}
Sep 11 19:24:11.017096 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:11.017056    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr" event={"ID":"603632b5-75d8-4672-8126-88b4b338c3b0","Type":"ContainerStarted","Data":"c1964b01a3614e4687ce49ad4f8bf88049c3eb4aa156d41a045f694a6b3ecda8"}
Sep 11 19:24:12.022474 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:12.022441    2583 generic.go:356] "Generic (PLEG): container finished" podID="603632b5-75d8-4672-8126-88b4b338c3b0" containerID="c1964b01a3614e4687ce49ad4f8bf88049c3eb4aa156d41a045f694a6b3ecda8" exitCode=0
Sep 11 19:24:12.022906 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:12.022537    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr" event={"ID":"603632b5-75d8-4672-8126-88b4b338c3b0","Type":"ContainerDied","Data":"c1964b01a3614e4687ce49ad4f8bf88049c3eb4aa156d41a045f694a6b3ecda8"}
Sep 11 19:24:12.834219 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:12.834139    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:12.935161 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:12.935122    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:13.028191 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:13.028158    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr" event={"ID":"603632b5-75d8-4672-8126-88b4b338c3b0","Type":"ContainerStarted","Data":"2223f57770982f372198c9a908846fab0bd58698d0d7409ee0c2117375b4e708"}
Sep 11 19:24:13.028573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:13.028289    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:13.070050 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:13.069999    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr" podStartSLOduration=3.331926703 podStartE2EDuration="8.069980578s" totalImagesPullingTime="3.735570064s" totalInitContainerRuntime="1.002483811s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:24:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:24:13.067787442 +0000 UTC m=+356.996412331" watchObservedRunningTime="2026-09-11 19:24:13.069980578 +0000 UTC m=+356.998605513"
Sep 11 19:24:20.692186 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:20.692155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:21.144392 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:21.144355    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:21.216480 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:21.216443    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:23.402041 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.402005    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"]
Sep 11 19:24:23.410827 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.410803    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.413891 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.413859    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 19:24:23.413891 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.413866    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 19:24:23.414092 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.413951    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 19:24:23.414092 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.414007    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9flmd\""
Sep 11 19:24:23.414823 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.414801    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"]
Sep 11 19:24:23.464607 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.464573    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d8e0559-c51c-4843-b808-a1440ed47c68-cert\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.464739 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.464617    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4d8e0559-c51c-4843-b808-a1440ed47c68-manager-config\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.464739 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.464688    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bczrf\" (UniqueName: \"kubernetes.io/projected/4d8e0559-c51c-4843-b808-a1440ed47c68-kube-api-access-bczrf\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.464739 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.464708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d8e0559-c51c-4843-b808-a1440ed47c68-metrics-cert\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.566201 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.566154    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bczrf\" (UniqueName: \"kubernetes.io/projected/4d8e0559-c51c-4843-b808-a1440ed47c68-kube-api-access-bczrf\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.566201 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.566202    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d8e0559-c51c-4843-b808-a1440ed47c68-metrics-cert\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.566475 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.566284    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d8e0559-c51c-4843-b808-a1440ed47c68-cert\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.566475 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.566328    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4d8e0559-c51c-4843-b808-a1440ed47c68-manager-config\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.567032 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.567006    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4d8e0559-c51c-4843-b808-a1440ed47c68-manager-config\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.568870 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.568847    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d8e0559-c51c-4843-b808-a1440ed47c68-cert\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.569010 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.568994    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d8e0559-c51c-4843-b808-a1440ed47c68-metrics-cert\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.574692 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.574668    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bczrf\" (UniqueName: \"kubernetes.io/projected/4d8e0559-c51c-4843-b808-a1440ed47c68-kube-api-access-bczrf\") pod \"lws-controller-manager-6df94f489d-dspkf\" (UID: \"4d8e0559-c51c-4843-b808-a1440ed47c68\") " pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.722083 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.722004    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:23.995736 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:23.995713    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"]
Sep 11 19:24:23.998063 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:24:23.998035    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8e0559_c51c_4843_b808_a1440ed47c68.slice/crio-315cbdad97f09ba2ab7255fa9ca82c1a53db1ac5ae7460ddde175e7aa14fe9dd WatchSource:0}: Error finding container 315cbdad97f09ba2ab7255fa9ca82c1a53db1ac5ae7460ddde175e7aa14fe9dd: Status 404 returned error can't find the container with id 315cbdad97f09ba2ab7255fa9ca82c1a53db1ac5ae7460ddde175e7aa14fe9dd
Sep 11 19:24:24.034089 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:24.034067    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6474799d6-hj8vr"
Sep 11 19:24:24.077885 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:24.077851    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf" event={"ID":"4d8e0559-c51c-4843-b808-a1440ed47c68","Type":"ContainerStarted","Data":"315cbdad97f09ba2ab7255fa9ca82c1a53db1ac5ae7460ddde175e7aa14fe9dd"}
Sep 11 19:24:24.386750 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:24.386718    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:24.494406 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:24.494373    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:26.088601 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:26.088513    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf" event={"ID":"4d8e0559-c51c-4843-b808-a1440ed47c68","Type":"ContainerStarted","Data":"5eeaef18c68ebff677c6a358076ebebdf56ac13c0c9ba5cc3c44da37ccec64bf"}
Sep 11 19:24:26.089063 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:26.088638    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:26.181426 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:26.181379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:26.211459 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:26.211406    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf" podStartSLOduration=1.602114613 podStartE2EDuration="3.211388738s" totalImagesPullingTime="1.609274125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:24:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:24:26.208942104 +0000 UTC m=+370.137567012" watchObservedRunningTime="2026-09-11 19:24:26.211388738 +0000 UTC m=+370.140013636"
Sep 11 19:24:26.404066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:26.404031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:31.889554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:31.889519    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:31.928992 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:31.928915    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-57c686645c-pshmw"]
Sep 11 19:24:31.932709 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:31.932688    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw"
Sep 11 19:24:31.935116 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:31.935088    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 19:24:31.935266 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:31.935169    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 19:24:31.935266 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:31.935192    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 19:24:31.935866 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:31.935848    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 19:24:31.935912 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:31.935853    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-b7w5t\""
Sep 11 19:24:31.944056 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:31.944031    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57c686645c-pshmw"]
Sep 11 19:24:32.040796 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.040762    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f330021-82db-481e-abfc-3d37913c42bb-tmp\") pod \"kube-auth-proxy-57c686645c-pshmw\" (UID: \"0f330021-82db-481e-abfc-3d37913c42bb\") " pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw"
Sep 11 19:24:32.040796 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.040801    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8nlx\" (UniqueName: \"kubernetes.io/projected/0f330021-82db-481e-abfc-3d37913c42bb-kube-api-access-s8nlx\") pod \"kube-auth-proxy-57c686645c-pshmw\" (UID: \"0f330021-82db-481e-abfc-3d37913c42bb\") " pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw"
Sep 11 19:24:32.040997 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.040868    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f330021-82db-481e-abfc-3d37913c42bb-tls-certs\") pod \"kube-auth-proxy-57c686645c-pshmw\" (UID: \"0f330021-82db-481e-abfc-3d37913c42bb\") " pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw"
Sep 11 19:24:32.141729 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.141630    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f330021-82db-481e-abfc-3d37913c42bb-tmp\") pod \"kube-auth-proxy-57c686645c-pshmw\" (UID: \"0f330021-82db-481e-abfc-3d37913c42bb\") " pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw"
Sep 11 19:24:32.141729 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.141685    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8nlx\" (UniqueName: \"kubernetes.io/projected/0f330021-82db-481e-abfc-3d37913c42bb-kube-api-access-s8nlx\") pod \"kube-auth-proxy-57c686645c-pshmw\" (UID: \"0f330021-82db-481e-abfc-3d37913c42bb\") " pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw"
Sep 11 19:24:32.141954 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.141755    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f330021-82db-481e-abfc-3d37913c42bb-tls-certs\") pod \"kube-auth-proxy-57c686645c-pshmw\" (UID: \"0f330021-82db-481e-abfc-3d37913c42bb\") " pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw"
Sep 11 19:24:32.142600 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.142545    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:32.144160 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.144141    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f330021-82db-481e-abfc-3d37913c42bb-tmp\") pod \"kube-auth-proxy-57c686645c-pshmw\" (UID: \"0f330021-82db-481e-abfc-3d37913c42bb\") " pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw"
Sep 11 19:24:32.144483 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.144461    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f330021-82db-481e-abfc-3d37913c42bb-tls-certs\") pod \"kube-auth-proxy-57c686645c-pshmw\" (UID: \"0f330021-82db-481e-abfc-3d37913c42bb\") " pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw"
Sep 11 19:24:32.150562 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.150532    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8nlx\" (UniqueName: \"kubernetes.io/projected/0f330021-82db-481e-abfc-3d37913c42bb-kube-api-access-s8nlx\") pod \"kube-auth-proxy-57c686645c-pshmw\" (UID: \"0f330021-82db-481e-abfc-3d37913c42bb\") " pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw"
Sep 11 19:24:32.242605 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.242573    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw"
Sep 11 19:24:32.515277 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:32.515193    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57c686645c-pshmw"]
Sep 11 19:24:32.517957 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:24:32.517927    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f330021_82db_481e_abfc_3d37913c42bb.slice/crio-c32c3c79283bea6affc5a6a59b3133f29cd38fbe025920ebd1546b413e67b483 WatchSource:0}: Error finding container c32c3c79283bea6affc5a6a59b3133f29cd38fbe025920ebd1546b413e67b483: Status 404 returned error can't find the container with id c32c3c79283bea6affc5a6a59b3133f29cd38fbe025920ebd1546b413e67b483
Sep 11 19:24:33.121060 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:33.121024    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw" event={"ID":"0f330021-82db-481e-abfc-3d37913c42bb","Type":"ContainerStarted","Data":"c32c3c79283bea6affc5a6a59b3133f29cd38fbe025920ebd1546b413e67b483"}
Sep 11 19:24:33.232699 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:33.232650    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:33.328228 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:33.328184    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:34.172588 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:34.172536    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:35.273759 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:35.273723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:35.374573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:35.374537    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:35.474510 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:35.474484    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:36.136430 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:36.136397    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw" event={"ID":"0f330021-82db-481e-abfc-3d37913c42bb","Type":"ContainerStarted","Data":"66aec6abb45d0fd1a4be2cc64d0809093bdf04b2cadc83988bb506669043cfb4"}
Sep 11 19:24:36.152478 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:36.152410    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-57c686645c-pshmw" podStartSLOduration=2.28374275 podStartE2EDuration="5.152392802s" totalImagesPullingTime="2.868650052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:24:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:24:36.151016338 +0000 UTC m=+380.079641235" watchObservedRunningTime="2026-09-11 19:24:36.152392802 +0000 UTC m=+380.081017746"
Sep 11 19:24:37.095408 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:37.095378    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6df94f489d-dspkf"
Sep 11 19:24:37.565742 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:37.565658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:37.667833 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:37.667791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:37.770366 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:37.770338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:38.030786 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:38.030752    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:38.143340 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:38.143310    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:38.350316 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:38.350277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:38.523104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:38.523067    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:39.046719 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:39.046687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:39.220696 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:39.220662    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:39.318314 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:39.318208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:39.809445 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:39.809399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:47.041504 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:47.041468    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:47.220795 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:47.220752    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:47.461411 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:47.461382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:48.078282 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:48.078233    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:48.419224 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:48.419192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:24:48.560151 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:24:48.560120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:38.251920 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:38.251844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:38.342837 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:38.342807    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:47.185800 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:47.185758    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:47.253469 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:47.253429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:53.868551 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:53.868522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:56.257872 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:56.257837    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:56.684500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:56.684463    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:58.090483 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:58.090449    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:58.192522 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:58.192486    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:59.060879 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:59.060849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:59.330005 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:59.329917    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:25:59.452746 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:25:59.452712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:00.070483 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:00.070457    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:00.902297 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:00.902267    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:03.481910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:03.481878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:03.562525 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:03.562485    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:06.480693 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:06.480653    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:06.604099 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:06.604057    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:10.170858 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.170821    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"]
Sep 11 19:26:10.173789 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.173764    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:10.174192 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.174175    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"
Sep 11 19:26:10.177201 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.177178    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 11 19:26:10.177318 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.177199    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 11 19:26:10.177318 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.177221    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-z9kkf\""
Sep 11 19:26:10.177954 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.177936    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 19:26:10.178019 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.177963    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 19:26:10.184659 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.184640    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"]
Sep 11 19:26:10.301997 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.301958    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrz6f\" (UniqueName: \"kubernetes.io/projected/dae641bf-fc1b-464f-ba54-c4452159b6c6-kube-api-access-wrz6f\") pod \"kuadrant-console-plugin-fbdd4f445-56j7m\" (UID: \"dae641bf-fc1b-464f-ba54-c4452159b6c6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"
Sep 11 19:26:10.302172 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.302017    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dae641bf-fc1b-464f-ba54-c4452159b6c6-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-56j7m\" (UID: \"dae641bf-fc1b-464f-ba54-c4452159b6c6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"
Sep 11 19:26:10.302172 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.302074    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dae641bf-fc1b-464f-ba54-c4452159b6c6-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-56j7m\" (UID: \"dae641bf-fc1b-464f-ba54-c4452159b6c6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"
Sep 11 19:26:10.402860 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.402826    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dae641bf-fc1b-464f-ba54-c4452159b6c6-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-56j7m\" (UID: \"dae641bf-fc1b-464f-ba54-c4452159b6c6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"
Sep 11 19:26:10.403045 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.402870    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrz6f\" (UniqueName: \"kubernetes.io/projected/dae641bf-fc1b-464f-ba54-c4452159b6c6-kube-api-access-wrz6f\") pod \"kuadrant-console-plugin-fbdd4f445-56j7m\" (UID: \"dae641bf-fc1b-464f-ba54-c4452159b6c6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"
Sep 11 19:26:10.403045 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.402906    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dae641bf-fc1b-464f-ba54-c4452159b6c6-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-56j7m\" (UID: \"dae641bf-fc1b-464f-ba54-c4452159b6c6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"
Sep 11 19:26:10.403594 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.403570    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dae641bf-fc1b-464f-ba54-c4452159b6c6-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-56j7m\" (UID: \"dae641bf-fc1b-464f-ba54-c4452159b6c6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"
Sep 11 19:26:10.405346 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.405329    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dae641bf-fc1b-464f-ba54-c4452159b6c6-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-56j7m\" (UID: \"dae641bf-fc1b-464f-ba54-c4452159b6c6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"
Sep 11 19:26:10.411885 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.411854    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrz6f\" (UniqueName: \"kubernetes.io/projected/dae641bf-fc1b-464f-ba54-c4452159b6c6-kube-api-access-wrz6f\") pod \"kuadrant-console-plugin-fbdd4f445-56j7m\" (UID: \"dae641bf-fc1b-464f-ba54-c4452159b6c6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"
Sep 11 19:26:10.433058 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.433000    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:10.483805 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.483778    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"
Sep 11 19:26:10.755651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:10.755497    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m"]
Sep 11 19:26:10.757852 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:26:10.757829    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddae641bf_fc1b_464f_ba54_c4452159b6c6.slice/crio-f4e6f2a25c63616605cd6de01b994d0bf662717d21cbe0ec35e09d56d67695a9 WatchSource:0}: Error finding container f4e6f2a25c63616605cd6de01b994d0bf662717d21cbe0ec35e09d56d67695a9: Status 404 returned error can't find the container with id f4e6f2a25c63616605cd6de01b994d0bf662717d21cbe0ec35e09d56d67695a9
Sep 11 19:26:11.539596 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:11.539548    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m" event={"ID":"dae641bf-fc1b-464f-ba54-c4452159b6c6","Type":"ContainerStarted","Data":"f4e6f2a25c63616605cd6de01b994d0bf662717d21cbe0ec35e09d56d67695a9"}
Sep 11 19:26:11.794202 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:11.794085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:11.896037 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:11.895954    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:11.996299 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:11.996268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:15.562134 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:15.562053    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m" event={"ID":"dae641bf-fc1b-464f-ba54-c4452159b6c6","Type":"ContainerStarted","Data":"78233c68ab9b5ad45e80035bc721ef4d5acfb8ea076ccd94b8d8eacd484988e0"}
Sep 11 19:26:15.578586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:15.578522    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-56j7m" podStartSLOduration=1.084520812 podStartE2EDuration="5.578506302s" totalImagesPullingTime="4.49398549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:26:15.577588267 +0000 UTC m=+479.506213165" watchObservedRunningTime="2026-09-11 19:26:15.578506302 +0000 UTC m=+479.507131202"
Sep 11 19:26:16.012471 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:16.012432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:16.109741 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:16.109686    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:16.210523 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:16.210489    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:21.355170 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.355136    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"]
Sep 11 19:26:21.359657 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.359632    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:21.362270 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.362233    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rddbf\""
Sep 11 19:26:21.382511 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.382480    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"]
Sep 11 19:26:21.511234 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.511197    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8989794f-d3a9-4d39-aa27-026fc818b08a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-blsvb\" (UID: \"8989794f-d3a9-4d39-aa27-026fc818b08a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:21.511454 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.511284    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb5qh\" (UniqueName: \"kubernetes.io/projected/8989794f-d3a9-4d39-aa27-026fc818b08a-kube-api-access-nb5qh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-blsvb\" (UID: \"8989794f-d3a9-4d39-aa27-026fc818b08a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:21.612264 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.612165    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8989794f-d3a9-4d39-aa27-026fc818b08a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-blsvb\" (UID: \"8989794f-d3a9-4d39-aa27-026fc818b08a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:21.612264 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.612206    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb5qh\" (UniqueName: \"kubernetes.io/projected/8989794f-d3a9-4d39-aa27-026fc818b08a-kube-api-access-nb5qh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-blsvb\" (UID: \"8989794f-d3a9-4d39-aa27-026fc818b08a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:21.612610 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.612592    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8989794f-d3a9-4d39-aa27-026fc818b08a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-blsvb\" (UID: \"8989794f-d3a9-4d39-aa27-026fc818b08a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:21.641436 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.641409    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb5qh\" (UniqueName: \"kubernetes.io/projected/8989794f-d3a9-4d39-aa27-026fc818b08a-kube-api-access-nb5qh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-blsvb\" (UID: \"8989794f-d3a9-4d39-aa27-026fc818b08a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:21.672306 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.672272    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:21.950707 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:21.950678    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"]
Sep 11 19:26:21.953747 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:26:21.953707    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8989794f_d3a9_4d39_aa27_026fc818b08a.slice/crio-f95a51797f81ba972ed8260a1ae34c12dcd9aa8e9e516dbceb4fdaa085bd4e7a WatchSource:0}: Error finding container f95a51797f81ba972ed8260a1ae34c12dcd9aa8e9e516dbceb4fdaa085bd4e7a: Status 404 returned error can't find the container with id f95a51797f81ba972ed8260a1ae34c12dcd9aa8e9e516dbceb4fdaa085bd4e7a
Sep 11 19:26:22.594453 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:22.594389    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb" event={"ID":"8989794f-d3a9-4d39-aa27-026fc818b08a","Type":"ContainerStarted","Data":"f95a51797f81ba972ed8260a1ae34c12dcd9aa8e9e516dbceb4fdaa085bd4e7a"}
Sep 11 19:26:23.113922 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:23.113879    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:23.213954 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:23.213896    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:25.704888 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:25.704848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:25.807303 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:25.807266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:26.613427 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:26.613387    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb" event={"ID":"8989794f-d3a9-4d39-aa27-026fc818b08a","Type":"ContainerStarted","Data":"3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd"}
Sep 11 19:26:26.613639 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:26.613528    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:26.634679 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:26.634618    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb" podStartSLOduration=1.615603262 podStartE2EDuration="5.634599747s" totalImagesPullingTime="4.018996485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:26:26.63327438 +0000 UTC m=+490.561899277" watchObservedRunningTime="2026-09-11 19:26:26.634599747 +0000 UTC m=+490.563224646"
Sep 11 19:26:31.087736 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:31.087691    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:31.188747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:31.188706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:31.486122 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:31.486078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:31.586997 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:31.586958    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:33.353393 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.353363    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"]
Sep 11 19:26:33.353824 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.353557    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb" podUID="8989794f-d3a9-4d39-aa27-026fc818b08a" containerName="manager" containerID="cri-o://3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd" gracePeriod=10
Sep 11 19:26:33.355293 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.355270    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:33.598763 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.598741    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:33.605543 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.605482    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8989794f-d3a9-4d39-aa27-026fc818b08a-extensions-socket-volume\") pod \"8989794f-d3a9-4d39-aa27-026fc818b08a\" (UID: \"8989794f-d3a9-4d39-aa27-026fc818b08a\") "
Sep 11 19:26:33.605660 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.605552    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nb5qh\" (UniqueName: \"kubernetes.io/projected/8989794f-d3a9-4d39-aa27-026fc818b08a-kube-api-access-nb5qh\") pod \"8989794f-d3a9-4d39-aa27-026fc818b08a\" (UID: \"8989794f-d3a9-4d39-aa27-026fc818b08a\") "
Sep 11 19:26:33.605877 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.605856    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8989794f-d3a9-4d39-aa27-026fc818b08a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8989794f-d3a9-4d39-aa27-026fc818b08a" (UID: "8989794f-d3a9-4d39-aa27-026fc818b08a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:26:33.607882 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.607858    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8989794f-d3a9-4d39-aa27-026fc818b08a-kube-api-access-nb5qh" (OuterVolumeSpecName: "kube-api-access-nb5qh") pod "8989794f-d3a9-4d39-aa27-026fc818b08a" (UID: "8989794f-d3a9-4d39-aa27-026fc818b08a"). InnerVolumeSpecName "kube-api-access-nb5qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:26:33.646335 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.646301    2583 generic.go:356] "Generic (PLEG): container finished" podID="8989794f-d3a9-4d39-aa27-026fc818b08a" containerID="3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd" exitCode=0
Sep 11 19:26:33.646500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.646346    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb" event={"ID":"8989794f-d3a9-4d39-aa27-026fc818b08a","Type":"ContainerDied","Data":"3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd"}
Sep 11 19:26:33.646500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.646351    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"
Sep 11 19:26:33.646500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.646388    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb" event={"ID":"8989794f-d3a9-4d39-aa27-026fc818b08a","Type":"ContainerDied","Data":"f95a51797f81ba972ed8260a1ae34c12dcd9aa8e9e516dbceb4fdaa085bd4e7a"}
Sep 11 19:26:33.646500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.646411    2583 scope.go:117] "RemoveContainer" containerID="3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd"
Sep 11 19:26:33.655513 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.655487    2583 scope.go:117] "RemoveContainer" containerID="3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd"
Sep 11 19:26:33.655782 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:26:33.655759    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd\": container with ID starting with 3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd not found: ID does not exist" containerID="3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd"
Sep 11 19:26:33.655866 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.655789    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd"} err="failed to get container status \"3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd\": rpc error: code = NotFound desc = could not find container \"3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd\": container with ID starting with 3dd1de41bfcf53717bf7739538d46368c41553f4eb131cf4ced203b51e0525fd not found: ID does not exist"
Sep 11 19:26:33.670005 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.669982    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"]
Sep 11 19:26:33.678811 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.678782    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-blsvb"]
Sep 11 19:26:33.706832 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.706803    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nb5qh\" (UniqueName: \"kubernetes.io/projected/8989794f-d3a9-4d39-aa27-026fc818b08a-kube-api-access-nb5qh\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:26:33.706832 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:33.706826    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8989794f-d3a9-4d39-aa27-026fc818b08a-extensions-socket-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:26:34.546429 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:34.546396    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8989794f-d3a9-4d39-aa27-026fc818b08a" path="/var/lib/kubelet/pods/8989794f-d3a9-4d39-aa27-026fc818b08a/volumes"
Sep 11 19:26:35.741104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:35.741068    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:35.842354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:35.842312    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:35.906996 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:35.906964    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:42.762550 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:42.762512    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:42.864883 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:42.864854    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:49.644780 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:49.644744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:50.012366 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:50.012281    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:51.883731 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.883685    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:51.896415 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.896385    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-v4spw"]
Sep 11 19:26:51.896697 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.896685    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8989794f-d3a9-4d39-aa27-026fc818b08a" containerName="manager"
Sep 11 19:26:51.896748 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.896725    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8989794f-d3a9-4d39-aa27-026fc818b08a" containerName="manager"
Sep 11 19:26:51.896822 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.896812    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8989794f-d3a9-4d39-aa27-026fc818b08a" containerName="manager"
Sep 11 19:26:51.914531 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.914504    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-v4spw"]
Sep 11 19:26:51.914668 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.914613    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:26:51.916713 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.916696    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 19:26:51.969352 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.969320    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d982e142-9410-4dd1-abc6-9cd8a6167591-config-file\") pod \"limitador-limitador-8588749977-v4spw\" (UID: \"d982e142-9410-4dd1-abc6-9cd8a6167591\") " pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:26:51.969523 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.969414    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g87qv\" (UniqueName: \"kubernetes.io/projected/d982e142-9410-4dd1-abc6-9cd8a6167591-kube-api-access-g87qv\") pod \"limitador-limitador-8588749977-v4spw\" (UID: \"d982e142-9410-4dd1-abc6-9cd8a6167591\") " pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:26:51.990563 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.990535    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-v4spw"]
Sep 11 19:26:51.993327 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:51.993297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:52.070517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.070481    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d982e142-9410-4dd1-abc6-9cd8a6167591-config-file\") pod \"limitador-limitador-8588749977-v4spw\" (UID: \"d982e142-9410-4dd1-abc6-9cd8a6167591\") " pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:26:52.070682 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.070536    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g87qv\" (UniqueName: \"kubernetes.io/projected/d982e142-9410-4dd1-abc6-9cd8a6167591-kube-api-access-g87qv\") pod \"limitador-limitador-8588749977-v4spw\" (UID: \"d982e142-9410-4dd1-abc6-9cd8a6167591\") " pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:26:52.071086 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.071064    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d982e142-9410-4dd1-abc6-9cd8a6167591-config-file\") pod \"limitador-limitador-8588749977-v4spw\" (UID: \"d982e142-9410-4dd1-abc6-9cd8a6167591\") " pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:26:52.078940 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.078918    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g87qv\" (UniqueName: \"kubernetes.io/projected/d982e142-9410-4dd1-abc6-9cd8a6167591-kube-api-access-g87qv\") pod \"limitador-limitador-8588749977-v4spw\" (UID: \"d982e142-9410-4dd1-abc6-9cd8a6167591\") " pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:26:52.094616 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.094585    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:52.225351 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.225275    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:26:52.497978 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.497951    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-v4spw"]
Sep 11 19:26:52.500643 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:26:52.500619    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd982e142_9410_4dd1_abc6_9cd8a6167591.slice/crio-2e5c7e0e83d537f9d1a5fa223c643a9a9efef6f189b71d0fd4e85df2d20f3625 WatchSource:0}: Error finding container 2e5c7e0e83d537f9d1a5fa223c643a9a9efef6f189b71d0fd4e85df2d20f3625: Status 404 returned error can't find the container with id 2e5c7e0e83d537f9d1a5fa223c643a9a9efef6f189b71d0fd4e85df2d20f3625
Sep 11 19:26:52.708370 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.708335    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-ccjcf"]
Sep 11 19:26:52.711315 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.711295    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf"
Sep 11 19:26:52.713466 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.713439    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nhrtd\""
Sep 11 19:26:52.720996 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.720976    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-ccjcf"]
Sep 11 19:26:52.727200 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.727176    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-v4spw" event={"ID":"d982e142-9410-4dd1-abc6-9cd8a6167591","Type":"ContainerStarted","Data":"2e5c7e0e83d537f9d1a5fa223c643a9a9efef6f189b71d0fd4e85df2d20f3625"}
Sep 11 19:26:52.741387 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.741355    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:52.779579 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.779489    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvld6\" (UniqueName: \"kubernetes.io/projected/32c48d2f-6fa3-46be-9410-2f640d9fabe1-kube-api-access-wvld6\") pod \"authorino-7bcfd8f9cd-ccjcf\" (UID: \"32c48d2f-6fa3-46be-9410-2f640d9fabe1\") " pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf"
Sep 11 19:26:52.821678 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.821645    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:26:52.880097 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.880061    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvld6\" (UniqueName: \"kubernetes.io/projected/32c48d2f-6fa3-46be-9410-2f640d9fabe1-kube-api-access-wvld6\") pod \"authorino-7bcfd8f9cd-ccjcf\" (UID: \"32c48d2f-6fa3-46be-9410-2f640d9fabe1\") " pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf"
Sep 11 19:26:52.888855 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:52.888823    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvld6\" (UniqueName: \"kubernetes.io/projected/32c48d2f-6fa3-46be-9410-2f640d9fabe1-kube-api-access-wvld6\") pod \"authorino-7bcfd8f9cd-ccjcf\" (UID: \"32c48d2f-6fa3-46be-9410-2f640d9fabe1\") " pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf"
Sep 11 19:26:53.021774 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:53.021736    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf"
Sep 11 19:26:53.288369 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:53.288218    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-ccjcf"]
Sep 11 19:26:53.290676 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:26:53.290643    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32c48d2f_6fa3_46be_9410_2f640d9fabe1.slice/crio-3e5684847ded0372046eb5ba63ebdc83592aab67ac7954d5783fb2564468989e WatchSource:0}: Error finding container 3e5684847ded0372046eb5ba63ebdc83592aab67ac7954d5783fb2564468989e: Status 404 returned error can't find the container with id 3e5684847ded0372046eb5ba63ebdc83592aab67ac7954d5783fb2564468989e
Sep 11 19:26:53.734211 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:53.734179    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf" event={"ID":"32c48d2f-6fa3-46be-9410-2f640d9fabe1","Type":"ContainerStarted","Data":"3e5684847ded0372046eb5ba63ebdc83592aab67ac7954d5783fb2564468989e"}
Sep 11 19:26:53.735617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:53.735588    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-v4spw" event={"ID":"d982e142-9410-4dd1-abc6-9cd8a6167591","Type":"ContainerStarted","Data":"4dcb90975911f169ecc7884ca7cc0a35729f7140776a8777558af5ada4a470f5"}
Sep 11 19:26:53.735730 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:53.735710    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:26:53.756729 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:53.756678    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-v4spw" podStartSLOduration=1.6071807009999999 podStartE2EDuration="2.756659583s" totalImagesPullingTime="1.149478882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:26:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:26:53.754543298 +0000 UTC m=+517.683168197" watchObservedRunningTime="2026-09-11 19:26:53.756659583 +0000 UTC m=+517.685284480"
Sep 11 19:26:55.747805 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:55.747760    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf" event={"ID":"32c48d2f-6fa3-46be-9410-2f640d9fabe1","Type":"ContainerStarted","Data":"c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04"}
Sep 11 19:26:55.784654 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:55.784605    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf" podStartSLOduration=1.964538429 podStartE2EDuration="3.784589065s" totalImagesPullingTime="1.820050636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:26:55.784321167 +0000 UTC m=+519.712946066" watchObservedRunningTime="2026-09-11 19:26:55.784589065 +0000 UTC m=+519.713213962"
Sep 11 19:26:56.476912 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:56.476879    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-ccjcf"]
Sep 11 19:26:57.757017 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:57.756977    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf" podUID="32c48d2f-6fa3-46be-9410-2f640d9fabe1" containerName="authorino" containerID="cri-o://c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04" gracePeriod=30
Sep 11 19:26:58.004355 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.004332    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf"
Sep 11 19:26:58.129532 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.129499    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvld6\" (UniqueName: \"kubernetes.io/projected/32c48d2f-6fa3-46be-9410-2f640d9fabe1-kube-api-access-wvld6\") pod \"32c48d2f-6fa3-46be-9410-2f640d9fabe1\" (UID: \"32c48d2f-6fa3-46be-9410-2f640d9fabe1\") "
Sep 11 19:26:58.131687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.131660    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c48d2f-6fa3-46be-9410-2f640d9fabe1-kube-api-access-wvld6" (OuterVolumeSpecName: "kube-api-access-wvld6") pod "32c48d2f-6fa3-46be-9410-2f640d9fabe1" (UID: "32c48d2f-6fa3-46be-9410-2f640d9fabe1"). InnerVolumeSpecName "kube-api-access-wvld6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:26:58.231095 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.231046    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvld6\" (UniqueName: \"kubernetes.io/projected/32c48d2f-6fa3-46be-9410-2f640d9fabe1-kube-api-access-wvld6\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:26:58.762975 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.762942    2583 generic.go:356] "Generic (PLEG): container finished" podID="32c48d2f-6fa3-46be-9410-2f640d9fabe1" containerID="c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04" exitCode=0
Sep 11 19:26:58.763441 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.762991    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf" event={"ID":"32c48d2f-6fa3-46be-9410-2f640d9fabe1","Type":"ContainerDied","Data":"c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04"}
Sep 11 19:26:58.763441 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.762997    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf"
Sep 11 19:26:58.763441 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.763038    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-ccjcf" event={"ID":"32c48d2f-6fa3-46be-9410-2f640d9fabe1","Type":"ContainerDied","Data":"3e5684847ded0372046eb5ba63ebdc83592aab67ac7954d5783fb2564468989e"}
Sep 11 19:26:58.763441 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.763054    2583 scope.go:117] "RemoveContainer" containerID="c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04"
Sep 11 19:26:58.771113 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.771096    2583 scope.go:117] "RemoveContainer" containerID="c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04"
Sep 11 19:26:58.771393 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:26:58.771376    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04\": container with ID starting with c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04 not found: ID does not exist" containerID="c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04"
Sep 11 19:26:58.771457 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.771400    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04"} err="failed to get container status \"c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04\": rpc error: code = NotFound desc = could not find container \"c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04\": container with ID starting with c5741f8187d93b0cf8b2ea5f5b826898f1c49548ae36ca6f43508c72fcf41d04 not found: ID does not exist"
Sep 11 19:26:58.777679 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.777657    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-ccjcf"]
Sep 11 19:26:58.781529 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:26:58.781509    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-ccjcf"]
Sep 11 19:27:00.546073 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:00.546041    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32c48d2f-6fa3-46be-9410-2f640d9fabe1" path="/var/lib/kubelet/pods/32c48d2f-6fa3-46be-9410-2f640d9fabe1/volumes"
Sep 11 19:27:04.741588 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:04.741548    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:27:05.466709 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:05.466672    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-v4spw"]
Sep 11 19:27:05.466918 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:05.466893    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-v4spw" podUID="d982e142-9410-4dd1-abc6-9cd8a6167591" containerName="limitador" containerID="cri-o://4dcb90975911f169ecc7884ca7cc0a35729f7140776a8777558af5ada4a470f5" gracePeriod=30
Sep 11 19:27:05.795813 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:05.795782    2583 generic.go:356] "Generic (PLEG): container finished" podID="d982e142-9410-4dd1-abc6-9cd8a6167591" containerID="4dcb90975911f169ecc7884ca7cc0a35729f7140776a8777558af5ada4a470f5" exitCode=0
Sep 11 19:27:05.796177 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:05.795854    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-v4spw" event={"ID":"d982e142-9410-4dd1-abc6-9cd8a6167591","Type":"ContainerDied","Data":"4dcb90975911f169ecc7884ca7cc0a35729f7140776a8777558af5ada4a470f5"}
Sep 11 19:27:06.408659 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.408634    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:27:06.505738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.505701    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g87qv\" (UniqueName: \"kubernetes.io/projected/d982e142-9410-4dd1-abc6-9cd8a6167591-kube-api-access-g87qv\") pod \"d982e142-9410-4dd1-abc6-9cd8a6167591\" (UID: \"d982e142-9410-4dd1-abc6-9cd8a6167591\") "
Sep 11 19:27:06.505936 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.505763    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d982e142-9410-4dd1-abc6-9cd8a6167591-config-file\") pod \"d982e142-9410-4dd1-abc6-9cd8a6167591\" (UID: \"d982e142-9410-4dd1-abc6-9cd8a6167591\") "
Sep 11 19:27:06.506123 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.506098    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d982e142-9410-4dd1-abc6-9cd8a6167591-config-file" (OuterVolumeSpecName: "config-file") pod "d982e142-9410-4dd1-abc6-9cd8a6167591" (UID: "d982e142-9410-4dd1-abc6-9cd8a6167591"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:27:06.508085 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.508064    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d982e142-9410-4dd1-abc6-9cd8a6167591-kube-api-access-g87qv" (OuterVolumeSpecName: "kube-api-access-g87qv") pod "d982e142-9410-4dd1-abc6-9cd8a6167591" (UID: "d982e142-9410-4dd1-abc6-9cd8a6167591"). InnerVolumeSpecName "kube-api-access-g87qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:27:06.606737 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.606704    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g87qv\" (UniqueName: \"kubernetes.io/projected/d982e142-9410-4dd1-abc6-9cd8a6167591-kube-api-access-g87qv\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:27:06.606737 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.606735    2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d982e142-9410-4dd1-abc6-9cd8a6167591-config-file\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:27:06.801302 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.801273    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-v4spw"
Sep 11 19:27:06.801754 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.801272    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-v4spw" event={"ID":"d982e142-9410-4dd1-abc6-9cd8a6167591","Type":"ContainerDied","Data":"2e5c7e0e83d537f9d1a5fa223c643a9a9efef6f189b71d0fd4e85df2d20f3625"}
Sep 11 19:27:06.801754 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.801402    2583 scope.go:117] "RemoveContainer" containerID="4dcb90975911f169ecc7884ca7cc0a35729f7140776a8777558af5ada4a470f5"
Sep 11 19:27:06.817009 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.816983    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-v4spw"]
Sep 11 19:27:06.820404 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:06.820380    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-v4spw"]
Sep 11 19:27:07.991984 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:07.991897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:08.052198 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.052168    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:08.405485 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.405453    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"]
Sep 11 19:27:08.405808 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.405796    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d982e142-9410-4dd1-abc6-9cd8a6167591" containerName="limitador"
Sep 11 19:27:08.405852 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.405812    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d982e142-9410-4dd1-abc6-9cd8a6167591" containerName="limitador"
Sep 11 19:27:08.405852 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.405845    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32c48d2f-6fa3-46be-9410-2f640d9fabe1" containerName="authorino"
Sep 11 19:27:08.405913 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.405854    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="32c48d2f-6fa3-46be-9410-2f640d9fabe1" containerName="authorino"
Sep 11 19:27:08.405945 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.405913    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d982e142-9410-4dd1-abc6-9cd8a6167591" containerName="limitador"
Sep 11 19:27:08.405945 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.405920    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="32c48d2f-6fa3-46be-9410-2f640d9fabe1" containerName="authorino"
Sep 11 19:27:08.408453 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.408435    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"
Sep 11 19:27:08.410814 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.410789    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 19:27:08.410942 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.410828    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 19:27:08.410942 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.410846    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 19:27:08.411544 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.411530    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-m7dqv\""
Sep 11 19:27:08.416551 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.416529    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"]
Sep 11 19:27:08.436788 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.436752    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:08.524352 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.524314    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86hwk\" (UniqueName: \"kubernetes.io/projected/1ff29e22-22fe-48a6-b73c-14b7273312c7-kube-api-access-86hwk\") pod \"postgres-69fb4594cf-bxbpq\" (UID: \"1ff29e22-22fe-48a6-b73c-14b7273312c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"
Sep 11 19:27:08.524533 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.524397    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1ff29e22-22fe-48a6-b73c-14b7273312c7-data\") pod \"postgres-69fb4594cf-bxbpq\" (UID: \"1ff29e22-22fe-48a6-b73c-14b7273312c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"
Sep 11 19:27:08.545668 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.545638    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d982e142-9410-4dd1-abc6-9cd8a6167591" path="/var/lib/kubelet/pods/d982e142-9410-4dd1-abc6-9cd8a6167591/volumes"
Sep 11 19:27:08.625064 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.625028    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1ff29e22-22fe-48a6-b73c-14b7273312c7-data\") pod \"postgres-69fb4594cf-bxbpq\" (UID: \"1ff29e22-22fe-48a6-b73c-14b7273312c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"
Sep 11 19:27:08.625218 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.625108    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86hwk\" (UniqueName: \"kubernetes.io/projected/1ff29e22-22fe-48a6-b73c-14b7273312c7-kube-api-access-86hwk\") pod \"postgres-69fb4594cf-bxbpq\" (UID: \"1ff29e22-22fe-48a6-b73c-14b7273312c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"
Sep 11 19:27:08.625446 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.625428    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1ff29e22-22fe-48a6-b73c-14b7273312c7-data\") pod \"postgres-69fb4594cf-bxbpq\" (UID: \"1ff29e22-22fe-48a6-b73c-14b7273312c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"
Sep 11 19:27:08.638563 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.638546    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86hwk\" (UniqueName: \"kubernetes.io/projected/1ff29e22-22fe-48a6-b73c-14b7273312c7-kube-api-access-86hwk\") pod \"postgres-69fb4594cf-bxbpq\" (UID: \"1ff29e22-22fe-48a6-b73c-14b7273312c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"
Sep 11 19:27:08.720390 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.720292    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"
Sep 11 19:27:08.890556 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.890523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:08.988521 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:08.988446    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"]
Sep 11 19:27:08.991981 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:27:08.991949    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ff29e22_22fe_48a6_b73c_14b7273312c7.slice/crio-e7d50f50cb2e9f3707e7d5a2c12138e319c75bddfe3b4df577c2e123059e0eca WatchSource:0}: Error finding container e7d50f50cb2e9f3707e7d5a2c12138e319c75bddfe3b4df577c2e123059e0eca: Status 404 returned error can't find the container with id e7d50f50cb2e9f3707e7d5a2c12138e319c75bddfe3b4df577c2e123059e0eca
Sep 11 19:27:09.816089 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:09.816023    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq" event={"ID":"1ff29e22-22fe-48a6-b73c-14b7273312c7","Type":"ContainerStarted","Data":"e7d50f50cb2e9f3707e7d5a2c12138e319c75bddfe3b4df577c2e123059e0eca"}
Sep 11 19:27:11.257858 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:11.257825    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:11.358663 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:11.358631    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:11.826584 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:11.826495    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq" event={"ID":"1ff29e22-22fe-48a6-b73c-14b7273312c7","Type":"ContainerStarted","Data":"b79d4ab8feff4783ce01e4a61e69c25cd6832206c13c808344a10b82aa16f7d4"}
Sep 11 19:27:11.826749 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:11.826614    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"
Sep 11 19:27:11.841896 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:11.841849    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq" podStartSLOduration=1.774656961 podStartE2EDuration="3.841836265s" totalImagesPullingTime="2.067179304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:27:11.840969535 +0000 UTC m=+535.769594444" watchObservedRunningTime="2026-09-11 19:27:11.841836265 +0000 UTC m=+535.770461163"
Sep 11 19:27:17.860329 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:17.860300    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bxbpq"
Sep 11 19:27:18.155884 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.155776    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:18.343862 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.343821    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:18.639780 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.639742    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c79c7d895-q5cwb"]
Sep 11 19:27:18.645579 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.645560    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c79c7d895-q5cwb"
Sep 11 19:27:18.648396 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.648363    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 19:27:18.648519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.648432    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nhrtd\""
Sep 11 19:27:18.649481 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.649455    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c79c7d895-q5cwb"]
Sep 11 19:27:18.808081 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.808048    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxkt2\" (UniqueName: \"kubernetes.io/projected/f27daebc-363e-4246-b532-dd0726876a5f-kube-api-access-mxkt2\") pod \"authorino-c79c7d895-q5cwb\" (UID: \"f27daebc-363e-4246-b532-dd0726876a5f\") " pod="kuadrant-system/authorino-c79c7d895-q5cwb"
Sep 11 19:27:18.808282 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.808092    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f27daebc-363e-4246-b532-dd0726876a5f-tls-cert\") pod \"authorino-c79c7d895-q5cwb\" (UID: \"f27daebc-363e-4246-b532-dd0726876a5f\") " pod="kuadrant-system/authorino-c79c7d895-q5cwb"
Sep 11 19:27:18.909044 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.908965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxkt2\" (UniqueName: \"kubernetes.io/projected/f27daebc-363e-4246-b532-dd0726876a5f-kube-api-access-mxkt2\") pod \"authorino-c79c7d895-q5cwb\" (UID: \"f27daebc-363e-4246-b532-dd0726876a5f\") " pod="kuadrant-system/authorino-c79c7d895-q5cwb"
Sep 11 19:27:18.909044 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.909010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f27daebc-363e-4246-b532-dd0726876a5f-tls-cert\") pod \"authorino-c79c7d895-q5cwb\" (UID: \"f27daebc-363e-4246-b532-dd0726876a5f\") " pod="kuadrant-system/authorino-c79c7d895-q5cwb"
Sep 11 19:27:18.911642 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.911624    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f27daebc-363e-4246-b532-dd0726876a5f-tls-cert\") pod \"authorino-c79c7d895-q5cwb\" (UID: \"f27daebc-363e-4246-b532-dd0726876a5f\") " pod="kuadrant-system/authorino-c79c7d895-q5cwb"
Sep 11 19:27:18.917650 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.917625    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxkt2\" (UniqueName: \"kubernetes.io/projected/f27daebc-363e-4246-b532-dd0726876a5f-kube-api-access-mxkt2\") pod \"authorino-c79c7d895-q5cwb\" (UID: \"f27daebc-363e-4246-b532-dd0726876a5f\") " pod="kuadrant-system/authorino-c79c7d895-q5cwb"
Sep 11 19:27:18.955579 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:18.955545    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c79c7d895-q5cwb"
Sep 11 19:27:19.224224 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:19.224194    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c79c7d895-q5cwb"]
Sep 11 19:27:19.226945 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:27:19.226915    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf27daebc_363e_4246_b532_dd0726876a5f.slice/crio-6259fc02c46633871c477a55d7b85d575ad3fd5a06367745f9ac09e1513c09bf WatchSource:0}: Error finding container 6259fc02c46633871c477a55d7b85d575ad3fd5a06367745f9ac09e1513c09bf: Status 404 returned error can't find the container with id 6259fc02c46633871c477a55d7b85d575ad3fd5a06367745f9ac09e1513c09bf
Sep 11 19:27:19.862801 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:19.862711    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c79c7d895-q5cwb" event={"ID":"f27daebc-363e-4246-b532-dd0726876a5f","Type":"ContainerStarted","Data":"0480c9f2136f231a73023ec81f011d7692ac6fd1b52116fe7b8e301f19ed4107"}
Sep 11 19:27:19.862801 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:19.862755    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c79c7d895-q5cwb" event={"ID":"f27daebc-363e-4246-b532-dd0726876a5f","Type":"ContainerStarted","Data":"6259fc02c46633871c477a55d7b85d575ad3fd5a06367745f9ac09e1513c09bf"}
Sep 11 19:27:19.877459 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:19.877413    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c79c7d895-q5cwb" podStartSLOduration=1.5050407749999999 podStartE2EDuration="1.877391545s" totalImagesPullingTime="372.35077ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:27:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:27:19.876774968 +0000 UTC m=+543.805399865" watchObservedRunningTime="2026-09-11 19:27:19.877391545 +0000 UTC m=+543.806016444"
Sep 11 19:27:24.589487 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.589425    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:24.729296 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.729235    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:24.824122 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.824072    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:24.849025 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.848995    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bfc579bd-7pmzm"]
Sep 11 19:27:24.857933 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.857909    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:24.860766 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.860743    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 19:27:24.860980 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.860964    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pchrp\""
Sep 11 19:27:24.861059 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.860964    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 19:27:24.861500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.861482    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 19:27:24.870045 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.870020    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-7pmzm"]
Sep 11 19:27:24.926490 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.926456    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:24.953641 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.953603    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-webhook-cert\") pod \"maas-controller-5bfc579bd-7pmzm\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") " pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:24.953641 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.953642    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-metrics-certs\") pod \"maas-controller-5bfc579bd-7pmzm\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") " pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:24.953843 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:24.953684    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rfsv\" (UniqueName: \"kubernetes.io/projected/d0e62692-ccbe-44ff-bb41-c262c8e44c67-kube-api-access-8rfsv\") pod \"maas-controller-5bfc579bd-7pmzm\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") " pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:25.054710 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:25.054677    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rfsv\" (UniqueName: \"kubernetes.io/projected/d0e62692-ccbe-44ff-bb41-c262c8e44c67-kube-api-access-8rfsv\") pod \"maas-controller-5bfc579bd-7pmzm\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") " pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:25.054882 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:25.054742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-webhook-cert\") pod \"maas-controller-5bfc579bd-7pmzm\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") " pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:25.054882 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:25.054762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-metrics-certs\") pod \"maas-controller-5bfc579bd-7pmzm\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") " pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:25.054882 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:27:25.054873    2583 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 11 19:27:25.054988 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:27:25.054926    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-metrics-certs podName:d0e62692-ccbe-44ff-bb41-c262c8e44c67 nodeName:}" failed. No retries permitted until 2026-09-11 19:27:25.554908385 +0000 UTC m=+549.483533262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-metrics-certs") pod "maas-controller-5bfc579bd-7pmzm" (UID: "d0e62692-ccbe-44ff-bb41-c262c8e44c67") : secret "maas-controller-metrics-tls" not found
Sep 11 19:27:25.057528 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:25.057502    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-webhook-cert\") pod \"maas-controller-5bfc579bd-7pmzm\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") " pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:25.065367 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:25.065346    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rfsv\" (UniqueName: \"kubernetes.io/projected/d0e62692-ccbe-44ff-bb41-c262c8e44c67-kube-api-access-8rfsv\") pod \"maas-controller-5bfc579bd-7pmzm\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") " pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:25.151160 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:25.151079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:25.559320 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:25.559196    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-metrics-certs\") pod \"maas-controller-5bfc579bd-7pmzm\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") " pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:25.561718 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:25.561692    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-metrics-certs\") pod \"maas-controller-5bfc579bd-7pmzm\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") " pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:25.768755 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:25.768698    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:26.040316 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:26.040262    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-7pmzm"]
Sep 11 19:27:26.042933 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:27:26.042910    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e62692_ccbe_44ff_bb41_c262c8e44c67.slice/crio-4c701af92352ae8cd042f1b45cee3351e48188d44fe1171c9989bc908f369a7d WatchSource:0}: Error finding container 4c701af92352ae8cd042f1b45cee3351e48188d44fe1171c9989bc908f369a7d: Status 404 returned error can't find the container with id 4c701af92352ae8cd042f1b45cee3351e48188d44fe1171c9989bc908f369a7d
Sep 11 19:27:26.895576 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:26.895536    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bfc579bd-7pmzm" event={"ID":"d0e62692-ccbe-44ff-bb41-c262c8e44c67","Type":"ContainerStarted","Data":"4c701af92352ae8cd042f1b45cee3351e48188d44fe1171c9989bc908f369a7d"}
Sep 11 19:27:27.762606 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:27.762571    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:28.395764 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:28.395732    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:28.494695 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:28.494659    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:27:28.906585 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:28.906555    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bfc579bd-7pmzm" event={"ID":"d0e62692-ccbe-44ff-bb41-c262c8e44c67","Type":"ContainerStarted","Data":"4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03"}
Sep 11 19:27:28.906789 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:28.906679    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:28.923778 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:28.923731    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bfc579bd-7pmzm" podStartSLOduration=2.695544374 podStartE2EDuration="4.923717117s" totalImagesPullingTime="2.228172743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:27:28.923184975 +0000 UTC m=+552.851809872" watchObservedRunningTime="2026-09-11 19:27:28.923717117 +0000 UTC m=+552.852342031"
Sep 11 19:27:39.913795 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:39.913762    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:27:59.729622 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:27:59.729545    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:00.058961 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:00.058838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:00.147630 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:00.147593    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:01.552696 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.552655    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:01.645038 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.644991    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:01.651764 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.651737    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"]
Sep 11 19:28:01.656289 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.656265    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.658770 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.658745    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 19:28:01.659353 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.659327    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5hbtr\""
Sep 11 19:28:01.659480 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.659382    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 19:28:01.668422 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.668398    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"]
Sep 11 19:28:01.755469 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.755441    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:01.775110 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.775072    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29834c3b-3aac-4dfa-b02b-8509d49ed264-openshift-service-ca\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.775332 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.775122    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-maas-api-tls\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.775332 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.775204    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-metrics-certs\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.775332 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.775323    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m28bs\" (UniqueName: \"kubernetes.io/projected/29834c3b-3aac-4dfa-b02b-8509d49ed264-kube-api-access-m28bs\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.876146 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.876108    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-maas-api-tls\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.876382 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.876188    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-metrics-certs\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.876382 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.876236    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m28bs\" (UniqueName: \"kubernetes.io/projected/29834c3b-3aac-4dfa-b02b-8509d49ed264-kube-api-access-m28bs\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.876382 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.876315    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29834c3b-3aac-4dfa-b02b-8509d49ed264-openshift-service-ca\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.876902 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.876879    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29834c3b-3aac-4dfa-b02b-8509d49ed264-openshift-service-ca\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.878851 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.878830    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-maas-api-tls\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.878905 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.878876    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-metrics-certs\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.883815 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.883795    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m28bs\" (UniqueName: \"kubernetes.io/projected/29834c3b-3aac-4dfa-b02b-8509d49ed264-kube-api-access-m28bs\") pod \"maas-api-f476c4797-7xxqg\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") " pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:01.966956 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:01.966925    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:02.130832 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.130757    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:02.234873 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.234838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:02.244648 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.244625    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"]
Sep 11 19:28:02.249677 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:28:02.249628    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29834c3b_3aac_4dfa_b02b_8509d49ed264.slice/crio-d8d073e1bb0b67e5441790aa73b4fa048aa51338d5a87bf8835b8f03beeae1e4 WatchSource:0}: Error finding container d8d073e1bb0b67e5441790aa73b4fa048aa51338d5a87bf8835b8f03beeae1e4: Status 404 returned error can't find the container with id d8d073e1bb0b67e5441790aa73b4fa048aa51338d5a87bf8835b8f03beeae1e4
Sep 11 19:28:02.264172 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.264148    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-6st4w"]
Sep 11 19:28:02.267520 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.267502    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w"
Sep 11 19:28:02.270106 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.270084    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 19:28:02.270210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.270127    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-xj27x\""
Sep 11 19:28:02.281210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.281187    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-6st4w"]
Sep 11 19:28:02.381560 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.381473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdvnh\" (UniqueName: \"kubernetes.io/projected/4a7b30ce-04ec-43b8-acff-7ce0680ca302-kube-api-access-jdvnh\") pod \"payload-processing-5b54b7fbb7-6st4w\" (UID: \"4a7b30ce-04ec-43b8-acff-7ce0680ca302\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w"
Sep 11 19:28:02.381560 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.381527    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a7b30ce-04ec-43b8-acff-7ce0680ca302-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-6st4w\" (UID: \"4a7b30ce-04ec-43b8-acff-7ce0680ca302\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w"
Sep 11 19:28:02.483094 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.483051    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdvnh\" (UniqueName: \"kubernetes.io/projected/4a7b30ce-04ec-43b8-acff-7ce0680ca302-kube-api-access-jdvnh\") pod \"payload-processing-5b54b7fbb7-6st4w\" (UID: \"4a7b30ce-04ec-43b8-acff-7ce0680ca302\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w"
Sep 11 19:28:02.483321 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.483117    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a7b30ce-04ec-43b8-acff-7ce0680ca302-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-6st4w\" (UID: \"4a7b30ce-04ec-43b8-acff-7ce0680ca302\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w"
Sep 11 19:28:02.483858 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.483834    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a7b30ce-04ec-43b8-acff-7ce0680ca302-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-6st4w\" (UID: \"4a7b30ce-04ec-43b8-acff-7ce0680ca302\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w"
Sep 11 19:28:02.496969 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.496933    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdvnh\" (UniqueName: \"kubernetes.io/projected/4a7b30ce-04ec-43b8-acff-7ce0680ca302-kube-api-access-jdvnh\") pod \"payload-processing-5b54b7fbb7-6st4w\" (UID: \"4a7b30ce-04ec-43b8-acff-7ce0680ca302\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w"
Sep 11 19:28:02.578016 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.577976    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w"
Sep 11 19:28:02.851500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:02.851334    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-6st4w"]
Sep 11 19:28:02.854860 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:28:02.854819    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a7b30ce_04ec_43b8_acff_7ce0680ca302.slice/crio-ebb57a2723e65aa090b691c8765447e19efa7dccce64936ee32691b1028d0f31 WatchSource:0}: Error finding container ebb57a2723e65aa090b691c8765447e19efa7dccce64936ee32691b1028d0f31: Status 404 returned error can't find the container with id ebb57a2723e65aa090b691c8765447e19efa7dccce64936ee32691b1028d0f31
Sep 11 19:28:03.019567 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:03.019515    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:03.071818 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:03.071769    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w" event={"ID":"4a7b30ce-04ec-43b8-acff-7ce0680ca302","Type":"ContainerStarted","Data":"ebb57a2723e65aa090b691c8765447e19efa7dccce64936ee32691b1028d0f31"}
Sep 11 19:28:03.073283 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:03.073221    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg" event={"ID":"29834c3b-3aac-4dfa-b02b-8509d49ed264","Type":"ContainerStarted","Data":"d8d073e1bb0b67e5441790aa73b4fa048aa51338d5a87bf8835b8f03beeae1e4"}
Sep 11 19:28:05.084116 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:05.084079    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w" event={"ID":"4a7b30ce-04ec-43b8-acff-7ce0680ca302","Type":"ContainerStarted","Data":"9618e854417b5c49e41252605ee390962efcc2dee1b46406e8341889f48fdafc"}
Sep 11 19:28:05.084782 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:05.084212    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w"
Sep 11 19:28:05.085748 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:05.085722    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg" event={"ID":"29834c3b-3aac-4dfa-b02b-8509d49ed264","Type":"ContainerStarted","Data":"00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9"}
Sep 11 19:28:05.085860 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:05.085849    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:05.102277 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:05.102220    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w" podStartSLOduration=1.724505968 podStartE2EDuration="3.102205995s" totalImagesPullingTime="1.377700027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:28:05.100923323 +0000 UTC m=+589.029548221" watchObservedRunningTime="2026-09-11 19:28:05.102205995 +0000 UTC m=+589.030830932"
Sep 11 19:28:05.119228 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:05.119179    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg" podStartSLOduration=2.480662208 podStartE2EDuration="4.119164298s" totalImagesPullingTime="1.63850209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:28:05.117915308 +0000 UTC m=+589.046540199" watchObservedRunningTime="2026-09-11 19:28:05.119164298 +0000 UTC m=+589.047789232"
Sep 11 19:28:11.096714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:11.096682    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:28:11.285818 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:11.285782    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:11.429518 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:11.429439    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:11.537351 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:11.537319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:12.102571 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:12.102535    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:12.200036 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:12.200003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:16.092863 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:16.092832    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-6st4w"
Sep 11 19:28:26.114284 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:26.114225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:28:55.718259 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:28:55.718211    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:29:11.429024 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:11.428955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:29:29.895640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:29.895605    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"]
Sep 11 19:29:29.898355 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:29.898328    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:29.901397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:29.901357    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 19:29:29.901397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:29.901386    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 19:29:29.901894 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:29.901357    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 11 19:29:29.901894 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:29.901875    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-phcxt\""
Sep 11 19:29:29.910399 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:29.910370    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"]
Sep 11 19:29:29.921847 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:29.921821    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:29:30.038747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.038707    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.038938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.038767    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.038938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.038787    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59565165-a6c6-4c39-b7a2-345d08c9b9a3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.038938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.038805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.038938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.038828    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.038938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.038891    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlrmc\" (UniqueName: \"kubernetes.io/projected/59565165-a6c6-4c39-b7a2-345d08c9b9a3-kube-api-access-wlrmc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.139523 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.139484    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59565165-a6c6-4c39-b7a2-345d08c9b9a3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.139523 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.139524    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.139773 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.139547    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.139773 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.139584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlrmc\" (UniqueName: \"kubernetes.io/projected/59565165-a6c6-4c39-b7a2-345d08c9b9a3-kube-api-access-wlrmc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.139773 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.139616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.139773 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.139651    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.140008 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.139967    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.140072 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.139992    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.140072 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.140044    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.141986 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.141963    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59565165-a6c6-4c39-b7a2-345d08c9b9a3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.142255 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.142222    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59565165-a6c6-4c39-b7a2-345d08c9b9a3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.147269 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.147198    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlrmc\" (UniqueName: \"kubernetes.io/projected/59565165-a6c6-4c39-b7a2-345d08c9b9a3-kube-api-access-wlrmc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qlb4q\" (UID: \"59565165-a6c6-4c39-b7a2-345d08c9b9a3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.216515 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.216485    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:30.486286 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.486261    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"]
Sep 11 19:29:30.488872 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:29:30.488842    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59565165_a6c6_4c39_b7a2_345d08c9b9a3.slice/crio-72031545a66eca70d5cb3255d51f5d1736a13ef664e41a7281ee844ce1a5d60a WatchSource:0}: Error finding container 72031545a66eca70d5cb3255d51f5d1736a13ef664e41a7281ee844ce1a5d60a: Status 404 returned error can't find the container with id 72031545a66eca70d5cb3255d51f5d1736a13ef664e41a7281ee844ce1a5d60a
Sep 11 19:29:30.490177 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:30.490153    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 19:29:31.486007 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:31.485960    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q" event={"ID":"59565165-a6c6-4c39-b7a2-345d08c9b9a3","Type":"ContainerStarted","Data":"72031545a66eca70d5cb3255d51f5d1736a13ef664e41a7281ee844ce1a5d60a"}
Sep 11 19:29:32.492235 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:32.492191    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q" event={"ID":"59565165-a6c6-4c39-b7a2-345d08c9b9a3","Type":"ContainerStarted","Data":"0234b743cafcbeb4f68179b0311645903964f9faf52c1c3045304d4fb57b36cf"}
Sep 11 19:29:32.509826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:32.509775    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q" podStartSLOduration=2.402713873 podStartE2EDuration="3.509758358s" totalImagesPullingTime="1.107044485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:29:32.507440305 +0000 UTC m=+676.436065202" watchObservedRunningTime="2026-09-11 19:29:32.509758358 +0000 UTC m=+676.438383256"
Sep 11 19:29:33.330801 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:33.330766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:29:40.217363 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:40.217331    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:40.217781 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:40.217383    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:40.229871 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:40.229842    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:40.538983 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:40.538902    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qlb4q"
Sep 11 19:29:41.335991 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:41.335954    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:29:44.200348 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.200311    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"]
Sep 11 19:29:44.208528 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.208504    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.210782 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.210752    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 11 19:29:44.214841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.214818    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"]
Sep 11 19:29:44.238834 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.238799    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:29:44.374579 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.374537    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.374833 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.374593    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.374833 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.374681    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.374833 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.374722    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40676762-202e-448a-ae0c-21b6417e0357-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.374833 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.374751    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p62lw\" (UniqueName: \"kubernetes.io/projected/40676762-202e-448a-ae0c-21b6417e0357-kube-api-access-p62lw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.374833 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.374790    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.475882 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.475800    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p62lw\" (UniqueName: \"kubernetes.io/projected/40676762-202e-448a-ae0c-21b6417e0357-kube-api-access-p62lw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.475882 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.475844    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.476317 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.475903    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.476317 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.475932    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.476317 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.475965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.476317 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.475990    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40676762-202e-448a-ae0c-21b6417e0357-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.476317 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.476307    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.476570 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.476332    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.476570 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.476489    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.478425 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.478405    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40676762-202e-448a-ae0c-21b6417e0357-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.478578 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.478563    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40676762-202e-448a-ae0c-21b6417e0357-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.483369 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.483346    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p62lw\" (UniqueName: \"kubernetes.io/projected/40676762-202e-448a-ae0c-21b6417e0357-kube-api-access-p62lw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj\" (UID: \"40676762-202e-448a-ae0c-21b6417e0357\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.520425 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.520382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:44.581460 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.581385    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:29:44.793727 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:44.793647    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"]
Sep 11 19:29:44.796828 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:29:44.796804    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40676762_202e_448a_ae0c_21b6417e0357.slice/crio-3c6a5cbb220c625290ace5f3fad1315bf4e75d2aedbce6869a8acbe9ebdc9054 WatchSource:0}: Error finding container 3c6a5cbb220c625290ace5f3fad1315bf4e75d2aedbce6869a8acbe9ebdc9054: Status 404 returned error can't find the container with id 3c6a5cbb220c625290ace5f3fad1315bf4e75d2aedbce6869a8acbe9ebdc9054
Sep 11 19:29:45.557822 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:45.557782    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj" event={"ID":"40676762-202e-448a-ae0c-21b6417e0357","Type":"ContainerStarted","Data":"cf5562ebb661366124ab7caa96982a9f3b40311cc5ca0780fc651c1c7a67c064"}
Sep 11 19:29:45.557822 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:45.557827    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj" event={"ID":"40676762-202e-448a-ae0c-21b6417e0357","Type":"ContainerStarted","Data":"3c6a5cbb220c625290ace5f3fad1315bf4e75d2aedbce6869a8acbe9ebdc9054"}
Sep 11 19:29:45.577255 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:45.577187    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj" podStartSLOduration=1.37871388 podStartE2EDuration="1.577171391s" totalImagesPullingTime="198.457511ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:29:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:29:45.575492891 +0000 UTC m=+689.504117788" watchObservedRunningTime="2026-09-11 19:29:45.577171391 +0000 UTC m=+689.505796289"
Sep 11 19:29:54.520791 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:54.520760    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:54.521197 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:54.521064    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:54.533528 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:54.533504    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:29:54.608365 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:29:54.608331    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj"
Sep 11 19:30:03.320816 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:03.320780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:03.424049 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:03.424014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:33.541591 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:33.541503    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c79c7d895-q5cwb"]
Sep 11 19:30:33.542156 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:33.541706    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c79c7d895-q5cwb" podUID="f27daebc-363e-4246-b532-dd0726876a5f" containerName="authorino" containerID="cri-o://0480c9f2136f231a73023ec81f011d7692ac6fd1b52116fe7b8e301f19ed4107" gracePeriod=30
Sep 11 19:30:33.770413 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:33.770377    2583 generic.go:356] "Generic (PLEG): container finished" podID="f27daebc-363e-4246-b532-dd0726876a5f" containerID="0480c9f2136f231a73023ec81f011d7692ac6fd1b52116fe7b8e301f19ed4107" exitCode=0
Sep 11 19:30:33.770572 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:33.770437    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c79c7d895-q5cwb" event={"ID":"f27daebc-363e-4246-b532-dd0726876a5f","Type":"ContainerDied","Data":"0480c9f2136f231a73023ec81f011d7692ac6fd1b52116fe7b8e301f19ed4107"}
Sep 11 19:30:33.797048 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:33.796987    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c79c7d895-q5cwb"
Sep 11 19:30:33.919860 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:33.919808    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f27daebc-363e-4246-b532-dd0726876a5f-tls-cert\") pod \"f27daebc-363e-4246-b532-dd0726876a5f\" (UID: \"f27daebc-363e-4246-b532-dd0726876a5f\") "
Sep 11 19:30:33.920045 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:33.919926    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxkt2\" (UniqueName: \"kubernetes.io/projected/f27daebc-363e-4246-b532-dd0726876a5f-kube-api-access-mxkt2\") pod \"f27daebc-363e-4246-b532-dd0726876a5f\" (UID: \"f27daebc-363e-4246-b532-dd0726876a5f\") "
Sep 11 19:30:33.922072 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:33.922046    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f27daebc-363e-4246-b532-dd0726876a5f-kube-api-access-mxkt2" (OuterVolumeSpecName: "kube-api-access-mxkt2") pod "f27daebc-363e-4246-b532-dd0726876a5f" (UID: "f27daebc-363e-4246-b532-dd0726876a5f"). InnerVolumeSpecName "kube-api-access-mxkt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:30:33.929758 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:33.929735    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f27daebc-363e-4246-b532-dd0726876a5f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f27daebc-363e-4246-b532-dd0726876a5f" (UID: "f27daebc-363e-4246-b532-dd0726876a5f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:30:34.021156 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:34.021122    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxkt2\" (UniqueName: \"kubernetes.io/projected/f27daebc-363e-4246-b532-dd0726876a5f-kube-api-access-mxkt2\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:30:34.021156 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:34.021150    2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f27daebc-363e-4246-b532-dd0726876a5f-tls-cert\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:30:34.053087 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:34.053008    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:34.148373 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:34.148337    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:34.776354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:34.776324    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c79c7d895-q5cwb"
Sep 11 19:30:34.776803 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:34.776369    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c79c7d895-q5cwb" event={"ID":"f27daebc-363e-4246-b532-dd0726876a5f","Type":"ContainerDied","Data":"6259fc02c46633871c477a55d7b85d575ad3fd5a06367745f9ac09e1513c09bf"}
Sep 11 19:30:34.776803 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:34.776422    2583 scope.go:117] "RemoveContainer" containerID="0480c9f2136f231a73023ec81f011d7692ac6fd1b52116fe7b8e301f19ed4107"
Sep 11 19:30:34.796214 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:34.796188    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c79c7d895-q5cwb"]
Sep 11 19:30:34.800184 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:34.800163    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c79c7d895-q5cwb"]
Sep 11 19:30:35.164077 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:35.164038    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:35.262491 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:35.262463    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:36.546538 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:36.546496    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f27daebc-363e-4246-b532-dd0726876a5f" path="/var/lib/kubelet/pods/f27daebc-363e-4246-b532-dd0726876a5f/volumes"
Sep 11 19:30:50.449212 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:50.449165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:50.547852 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:50.547806    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:50.649534 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:50.649499    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:50.789937 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:50.789853    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:50.878648 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:50.878613    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:51.012345 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:51.012313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:51.137719 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:51.137670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:51.237588 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:51.237533    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:51.559510 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:51.559376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:51.902546 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:51.902461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:51.998800 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:51.998766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:52.343434 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.343404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:52.484360 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.484327    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7810-456f26"]
Sep 11 19:30:52.484668 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.484657    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f27daebc-363e-4246-b532-dd0726876a5f" containerName="authorino"
Sep 11 19:30:52.484712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.484670    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f27daebc-363e-4246-b532-dd0726876a5f" containerName="authorino"
Sep 11 19:30:52.484747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.484738    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f27daebc-363e-4246-b532-dd0726876a5f" containerName="authorino"
Sep 11 19:30:52.487770 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.487754    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7810-456f26"
Sep 11 19:30:52.489840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.489821    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-v69wf\""
Sep 11 19:30:52.493169 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.493146    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7810-456f26"]
Sep 11 19:30:52.578566 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.578531    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7ncx\" (UniqueName: \"kubernetes.io/projected/b4854933-c4e8-437a-aacf-9d1c71482808-kube-api-access-p7ncx\") pod \"test-curl-7810-456f26\" (UID: \"b4854933-c4e8-437a-aacf-9d1c71482808\") " pod="opendatahub/test-curl-7810-456f26"
Sep 11 19:30:52.679652 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.679550    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7ncx\" (UniqueName: \"kubernetes.io/projected/b4854933-c4e8-437a-aacf-9d1c71482808-kube-api-access-p7ncx\") pod \"test-curl-7810-456f26\" (UID: \"b4854933-c4e8-437a-aacf-9d1c71482808\") " pod="opendatahub/test-curl-7810-456f26"
Sep 11 19:30:52.690412 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.690386    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7ncx\" (UniqueName: \"kubernetes.io/projected/b4854933-c4e8-437a-aacf-9d1c71482808-kube-api-access-p7ncx\") pod \"test-curl-7810-456f26\" (UID: \"b4854933-c4e8-437a-aacf-9d1c71482808\") " pod="opendatahub/test-curl-7810-456f26"
Sep 11 19:30:52.798832 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:52.798795    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7810-456f26"
Sep 11 19:30:53.070026 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:53.069930    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7810-456f26"]
Sep 11 19:30:53.073753 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:30:53.073724    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4854933_c4e8_437a_aacf_9d1c71482808.slice/crio-38d1bce5dbb8f5ee5f8b3cede04a375291fd4f0bf031670fa862a5020f0c4fc6 WatchSource:0}: Error finding container 38d1bce5dbb8f5ee5f8b3cede04a375291fd4f0bf031670fa862a5020f0c4fc6: Status 404 returned error can't find the container with id 38d1bce5dbb8f5ee5f8b3cede04a375291fd4f0bf031670fa862a5020f0c4fc6
Sep 11 19:30:53.871263 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:53.871218    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7810-456f26" event={"ID":"b4854933-c4e8-437a-aacf-9d1c71482808","Type":"ContainerStarted","Data":"38d1bce5dbb8f5ee5f8b3cede04a375291fd4f0bf031670fa862a5020f0c4fc6"}
Sep 11 19:30:54.107409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:54.107368    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:54.213299 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:54.213143    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:54.300541 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:54.300500    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:55.242098 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.242065    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"]
Sep 11 19:30:55.253980 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.253515    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.255596 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.255526    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"]
Sep 11 19:30:55.255724 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.255613    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a0610b-serving-cert\""
Sep 11 19:30:55.304738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.304708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-metrics-certs\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.304905 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.304752    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2797c\" (UniqueName: \"kubernetes.io/projected/77daedc1-6d88-4c54-a00a-d6039d9474c9-kube-api-access-2797c\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.304961 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.304895    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.304961 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.304937    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77daedc1-6d88-4c54-a00a-d6039d9474c9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.319837 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.319614    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:55.406229 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.406194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-metrics-certs\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.406229 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.406236    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2797c\" (UniqueName: \"kubernetes.io/projected/77daedc1-6d88-4c54-a00a-d6039d9474c9-kube-api-access-2797c\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.406459 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.406296    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.406459 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.406313    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77daedc1-6d88-4c54-a00a-d6039d9474c9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.406459 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:30:55.406401    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-serving-cert: secret "maas-api-e2e-inf-a-a0610b-serving-cert" not found
Sep 11 19:30:55.406459 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:30:55.406458    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-maas-api-tls podName:77daedc1-6d88-4c54-a00a-d6039d9474c9 nodeName:}" failed. No retries permitted until 2026-09-11 19:30:55.906442774 +0000 UTC m=+759.835067663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-maas-api-tls") pod "maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m" (UID: "77daedc1-6d88-4c54-a00a-d6039d9474c9") : secret "maas-api-e2e-inf-a-a0610b-serving-cert" not found
Sep 11 19:30:55.407070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.407003    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77daedc1-6d88-4c54-a00a-d6039d9474c9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.408798 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.408777    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-metrics-certs\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.417833 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.417776    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2797c\" (UniqueName: \"kubernetes.io/projected/77daedc1-6d88-4c54-a00a-d6039d9474c9-kube-api-access-2797c\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.418893 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.418857    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:55.678946 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.678903    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:55.883505 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.883461    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7810-456f26" event={"ID":"b4854933-c4e8-437a-aacf-9d1c71482808","Type":"ContainerStarted","Data":"913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e"}
Sep 11 19:30:55.899872 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.899823    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7810-456f26" podStartSLOduration=1.485419422 podStartE2EDuration="3.899806831s" totalImagesPullingTime="2.414387409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:30:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:30:55.899292398 +0000 UTC m=+759.827917297" watchObservedRunningTime="2026-09-11 19:30:55.899806831 +0000 UTC m=+759.828431728"
Sep 11 19:30:55.911430 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.911398    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.914253 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.914213    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:55.948403 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:55.948369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:56.043067 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.043037    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:56.051139 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.051109    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"]
Sep 11 19:30:56.054549 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.054533    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:30:56.056696 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.056673    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-a0610b-dockercfg-ztsjv\""
Sep 11 19:30:56.056820 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.056804    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-a0610b\""
Sep 11 19:30:56.066579 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.066553    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"]
Sep 11 19:30:56.078716 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.078683    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"]
Sep 11 19:30:56.082234 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.082211    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:30:56.094105 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.093389    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"]
Sep 11 19:30:56.113214 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.113181    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d14a765-b470-4422-88d0-a7dd292f277f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr\" (UID: \"0d14a765-b470-4422-88d0-a7dd292f277f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:30:56.113377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.113256    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsf9k\" (UniqueName: \"kubernetes.io/projected/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-kube-api-access-nsf9k\") pod \"payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2\" (UID: \"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:30:56.113377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.113289    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j684b\" (UniqueName: \"kubernetes.io/projected/0d14a765-b470-4422-88d0-a7dd292f277f-kube-api-access-j684b\") pod \"payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr\" (UID: \"0d14a765-b470-4422-88d0-a7dd292f277f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:30:56.113377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.113334    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2\" (UID: \"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:30:56.168810 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.168727    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:56.214333 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.214290    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2\" (UID: \"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:30:56.214527 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.214426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d14a765-b470-4422-88d0-a7dd292f277f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr\" (UID: \"0d14a765-b470-4422-88d0-a7dd292f277f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:30:56.214527 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.214460    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsf9k\" (UniqueName: \"kubernetes.io/projected/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-kube-api-access-nsf9k\") pod \"payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2\" (UID: \"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:30:56.214527 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.214476    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j684b\" (UniqueName: \"kubernetes.io/projected/0d14a765-b470-4422-88d0-a7dd292f277f-kube-api-access-j684b\") pod \"payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr\" (UID: \"0d14a765-b470-4422-88d0-a7dd292f277f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:30:56.215136 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.215112    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2\" (UID: \"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:30:56.215300 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.215283    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d14a765-b470-4422-88d0-a7dd292f277f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr\" (UID: \"0d14a765-b470-4422-88d0-a7dd292f277f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:30:56.224829 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.224739    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j684b\" (UniqueName: \"kubernetes.io/projected/0d14a765-b470-4422-88d0-a7dd292f277f-kube-api-access-j684b\") pod \"payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr\" (UID: \"0d14a765-b470-4422-88d0-a7dd292f277f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:30:56.226125 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.226049    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsf9k\" (UniqueName: \"kubernetes.io/projected/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-kube-api-access-nsf9k\") pod \"payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2\" (UID: \"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:30:56.366044 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.366011    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:30:56.393362 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.393329    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:30:56.414647 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.414617    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"]
Sep 11 19:30:56.418523 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.418448    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.421150 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.421112    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-8fbb58-serving-cert\""
Sep 11 19:30:56.428666 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.428593    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"]
Sep 11 19:30:56.451215 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.451178    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"]
Sep 11 19:30:56.452666 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:30:56.452614    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77daedc1_6d88_4c54_a00a_d6039d9474c9.slice/crio-71437acf544e5f101c2c1d15baec3125769e75edd5c93d6d616d72cd2fbdd06b WatchSource:0}: Error finding container 71437acf544e5f101c2c1d15baec3125769e75edd5c93d6d616d72cd2fbdd06b: Status 404 returned error can't find the container with id 71437acf544e5f101c2c1d15baec3125769e75edd5c93d6d616d72cd2fbdd06b
Sep 11 19:30:56.484124 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.484092    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7810-456f26"]
Sep 11 19:30:56.489822 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.489793    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7810-456f26"]
Sep 11 19:30:56.513570 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.513536    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:56.516715 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.516691    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-metrics-certs\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.516863 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.516727    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26c66035-04c3-4037-b693-88ff86d1f870-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.516863 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.516785    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dwh5\" (UniqueName: \"kubernetes.io/projected/26c66035-04c3-4037-b693-88ff86d1f870-kube-api-access-8dwh5\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.516863 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.516805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-maas-api-tls\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.602344 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.602309    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:56.618125 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.618085    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-metrics-certs\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.618410 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.618383    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26c66035-04c3-4037-b693-88ff86d1f870-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.618530 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.618493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dwh5\" (UniqueName: \"kubernetes.io/projected/26c66035-04c3-4037-b693-88ff86d1f870-kube-api-access-8dwh5\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.618593 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.618529    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-maas-api-tls\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.618771 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:30:56.618707    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-serving-cert: secret "maas-api-e2e-worker-w4-8fbb58-serving-cert" not found
Sep 11 19:30:56.618880 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:30:56.618801    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-maas-api-tls podName:26c66035-04c3-4037-b693-88ff86d1f870 nodeName:}" failed. No retries permitted until 2026-09-11 19:30:57.118781547 +0000 UTC m=+761.047406425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-maas-api-tls") pod "maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd" (UID: "26c66035-04c3-4037-b693-88ff86d1f870") : secret "maas-api-e2e-worker-w4-8fbb58-serving-cert" not found
Sep 11 19:30:56.619321 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.619298    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26c66035-04c3-4037-b693-88ff86d1f870-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.621089 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.621067    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-metrics-certs\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.628943 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.628921    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dwh5\" (UniqueName: \"kubernetes.io/projected/26c66035-04c3-4037-b693-88ff86d1f870-kube-api-access-8dwh5\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:56.645519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.645489    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"]
Sep 11 19:30:56.682069 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.682047    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"]
Sep 11 19:30:56.684434 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:30:56.684407    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e5a0bd_966f_4fcb_b5b9_fe630cf0edaf.slice/crio-fbccd1d851652f5694de6b893257bc1f1053a15a507ec422793fb0456fb6a401 WatchSource:0}: Error finding container fbccd1d851652f5694de6b893257bc1f1053a15a507ec422793fb0456fb6a401: Status 404 returned error can't find the container with id fbccd1d851652f5694de6b893257bc1f1053a15a507ec422793fb0456fb6a401
Sep 11 19:30:56.790060 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.790025    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:56.891441 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.891398    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m" event={"ID":"77daedc1-6d88-4c54-a00a-d6039d9474c9","Type":"ContainerStarted","Data":"71437acf544e5f101c2c1d15baec3125769e75edd5c93d6d616d72cd2fbdd06b"}
Sep 11 19:30:56.893508 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.893474    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2" event={"ID":"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf","Type":"ContainerStarted","Data":"3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce"}
Sep 11 19:30:56.893684 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.893535    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2" event={"ID":"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf","Type":"ContainerStarted","Data":"fbccd1d851652f5694de6b893257bc1f1053a15a507ec422793fb0456fb6a401"}
Sep 11 19:30:56.893684 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.893578    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:30:56.895358 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.895327    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr" event={"ID":"0d14a765-b470-4422-88d0-a7dd292f277f","Type":"ContainerStarted","Data":"18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed"}
Sep 11 19:30:56.895466 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.895385    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr" event={"ID":"0d14a765-b470-4422-88d0-a7dd292f277f","Type":"ContainerStarted","Data":"d5013cf8dcda816679dcaa11fddc9d611f64d8b56e98617a1deaa8c61cbd12ac"}
Sep 11 19:30:56.895466 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.895416    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:30:56.895558 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:30:56.895512    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7810-456f26" container="test-curl-7810-456f26"
Sep 11 19:30:56.918835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.918767    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2" podStartSLOduration=0.918740576 podStartE2EDuration="918.740576ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:30:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:30:56.912346675 +0000 UTC m=+760.840971577" watchObservedRunningTime="2026-09-11 19:30:56.918740576 +0000 UTC m=+760.847365472"
Sep 11 19:30:56.939221 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:56.939116    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr" podStartSLOduration=0.939098087 podStartE2EDuration="939.098087ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:30:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:30:56.935158543 +0000 UTC m=+760.863783440" watchObservedRunningTime="2026-09-11 19:30:56.939098087 +0000 UTC m=+760.867722985"
Sep 11 19:30:57.125222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.125186    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-maas-api-tls\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:57.127906 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.127881    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-maas-api-tls\") pod \"maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:57.200620 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.200536    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:57.299164 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.299133    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:57.344228 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.344198    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:57.346004 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.345979    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"]
Sep 11 19:30:57.346606 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.346585    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4854933-c4e8-437a-aacf-9d1c71482808" containerName="test-curl-7810-456f26"
Sep 11 19:30:57.346760 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.346748    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4854933-c4e8-437a-aacf-9d1c71482808" containerName="test-curl-7810-456f26"
Sep 11 19:30:57.346935 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.346924    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4854933-c4e8-437a-aacf-9d1c71482808" containerName="test-curl-7810-456f26"
Sep 11 19:30:57.350416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.350393    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:30:57.353118 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.353096    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-8fbb58\""
Sep 11 19:30:57.353304 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.353282    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-8fbb58-dockercfg-h6qt7\""
Sep 11 19:30:57.362069 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.362042    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"]
Sep 11 19:30:57.428384 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.428314    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc6dc\" (UniqueName: \"kubernetes.io/projected/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-kube-api-access-nc6dc\") pod \"payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6\" (UID: \"78d4b411-e77d-4693-8aff-fc2d20dfd9c5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:30:57.428810 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.428460    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6\" (UID: \"78d4b411-e77d-4693-8aff-fc2d20dfd9c5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:30:57.529541 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.529447    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6\" (UID: \"78d4b411-e77d-4693-8aff-fc2d20dfd9c5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:30:57.529541 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.529533    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc6dc\" (UniqueName: \"kubernetes.io/projected/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-kube-api-access-nc6dc\") pod \"payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6\" (UID: \"78d4b411-e77d-4693-8aff-fc2d20dfd9c5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:30:57.530160 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.530139    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6\" (UID: \"78d4b411-e77d-4693-8aff-fc2d20dfd9c5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:30:57.537666 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.537643    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc6dc\" (UniqueName: \"kubernetes.io/projected/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-kube-api-access-nc6dc\") pod \"payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6\" (UID: \"78d4b411-e77d-4693-8aff-fc2d20dfd9c5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:30:57.603845 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.603803    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"]
Sep 11 19:30:57.609141 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.609116    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.611842 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.611586    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-1fc808-serving-cert\""
Sep 11 19:30:57.619098 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.618662    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"]
Sep 11 19:30:57.642235 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.642154    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"]
Sep 11 19:30:57.644557 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.644524    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:57.645891 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:30:57.645858    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26c66035_04c3_4037_b693_88ff86d1f870.slice/crio-7e458b9908de7ff835d253f7103ab6c6110f600677bed9c42c9853602a7ac626 WatchSource:0}: Error finding container 7e458b9908de7ff835d253f7103ab6c6110f600677bed9c42c9853602a7ac626: Status 404 returned error can't find the container with id 7e458b9908de7ff835d253f7103ab6c6110f600677bed9c42c9853602a7ac626
Sep 11 19:30:57.666841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.666816    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:30:57.730778 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.730742    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.730778 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.730791    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sptd\" (UniqueName: \"kubernetes.io/projected/a5203fea-7020-4559-b071-36df336e300b-kube-api-access-5sptd\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.731016 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.730819    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5203fea-7020-4559-b071-36df336e300b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.731016 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.730934    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-metrics-certs\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.750221 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.750178    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:57.831737 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.831646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5203fea-7020-4559-b071-36df336e300b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.831737 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.831693    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-metrics-certs\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.831927 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.831786    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.831927 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.831818    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sptd\" (UniqueName: \"kubernetes.io/projected/a5203fea-7020-4559-b071-36df336e300b-kube-api-access-5sptd\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.832452 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.832427    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5203fea-7020-4559-b071-36df336e300b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.834399 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.834381    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-metrics-certs\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.834489 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.834430    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.839558 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.839531    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sptd\" (UniqueName: \"kubernetes.io/projected/a5203fea-7020-4559-b071-36df336e300b-kube-api-access-5sptd\") pod \"maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.844205 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.844183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:57.902418 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.902384    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd" event={"ID":"26c66035-04c3-4037-b693-88ff86d1f870","Type":"ContainerStarted","Data":"7e458b9908de7ff835d253f7103ab6c6110f600677bed9c42c9853602a7ac626"}
Sep 11 19:30:57.904180 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.904146    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m" event={"ID":"77daedc1-6d88-4c54-a00a-d6039d9474c9","Type":"ContainerStarted","Data":"b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a"}
Sep 11 19:30:57.904180 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.904175    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7810-456f26" podUID="b4854933-c4e8-437a-aacf-9d1c71482808" containerName="test-curl-7810-456f26" containerID="cri-o://913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e" gracePeriod=2
Sep 11 19:30:57.922120 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.922066    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m" podStartSLOduration=2.535504295 podStartE2EDuration="2.922047202s" totalImagesPullingTime="386.542907ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:30:57.9201344 +0000 UTC m=+761.848759299" watchObservedRunningTime="2026-09-11 19:30:57.922047202 +0000 UTC m=+761.850672121"
Sep 11 19:30:57.925381 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.925361    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:57.941517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:57.941492    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"]
Sep 11 19:30:57.943995 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:30:57.943966    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78d4b411_e77d_4693_8aff_fc2d20dfd9c5.slice/crio-3deafdea69c3a843efb00adce55c9a876aa0b67ac1c085c9cbfa60a97a683aec WatchSource:0}: Error finding container 3deafdea69c3a843efb00adce55c9a876aa0b67ac1c085c9cbfa60a97a683aec: Status 404 returned error can't find the container with id 3deafdea69c3a843efb00adce55c9a876aa0b67ac1c085c9cbfa60a97a683aec
Sep 11 19:30:58.144195 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.144165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:58.216622 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.216586    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"]
Sep 11 19:30:58.219251 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:30:58.219209    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5203fea_7020_4559_b071_36df336e300b.slice/crio-d8b2127f0664cff7d885992545280bed14f4b5043dc7dac2bf101e76d7671082 WatchSource:0}: Error finding container d8b2127f0664cff7d885992545280bed14f4b5043dc7dac2bf101e76d7671082: Status 404 returned error can't find the container with id d8b2127f0664cff7d885992545280bed14f4b5043dc7dac2bf101e76d7671082
Sep 11 19:30:58.224841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.224808    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:58.705803 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.705769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:58.912864 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.912822    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd" event={"ID":"26c66035-04c3-4037-b693-88ff86d1f870","Type":"ContainerStarted","Data":"aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3"}
Sep 11 19:30:58.913038 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.912979    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:30:58.914546 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.914522    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh" event={"ID":"a5203fea-7020-4559-b071-36df336e300b","Type":"ContainerStarted","Data":"3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9"}
Sep 11 19:30:58.914680 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.914567    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh" event={"ID":"a5203fea-7020-4559-b071-36df336e300b","Type":"ContainerStarted","Data":"d8b2127f0664cff7d885992545280bed14f4b5043dc7dac2bf101e76d7671082"}
Sep 11 19:30:58.914680 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.914631    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:30:58.916384 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.916361    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6" event={"ID":"78d4b411-e77d-4693-8aff-fc2d20dfd9c5","Type":"ContainerStarted","Data":"577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f"}
Sep 11 19:30:58.916515 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.916394    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6" event={"ID":"78d4b411-e77d-4693-8aff-fc2d20dfd9c5","Type":"ContainerStarted","Data":"3deafdea69c3a843efb00adce55c9a876aa0b67ac1c085c9cbfa60a97a683aec"}
Sep 11 19:30:58.916574 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.916538    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:30:58.916574 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.916556    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:30:58.930271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.930202    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd" podStartSLOduration=2.506548894 podStartE2EDuration="2.930184129s" totalImagesPullingTime="423.635235ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:30:58.929753592 +0000 UTC m=+762.858378482" watchObservedRunningTime="2026-09-11 19:30:58.930184129 +0000 UTC m=+762.858809024"
Sep 11 19:30:58.947587 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:58.947540    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh" podStartSLOduration=1.620518763 podStartE2EDuration="1.9475234s" totalImagesPullingTime="327.004637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:30:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:30:58.946079785 +0000 UTC m=+762.874704695" watchObservedRunningTime="2026-09-11 19:30:58.9475234 +0000 UTC m=+762.876148297"
Sep 11 19:30:59.370079 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:59.370034    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:59.472521 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:59.472488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:59.570129 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:59.570101    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:59.659889 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:59.659812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:30:59.926373 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:30:59.926329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:00.088160 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.088132    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7810-456f26"
Sep 11 19:31:00.153937 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.153901    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7ncx\" (UniqueName: \"kubernetes.io/projected/b4854933-c4e8-437a-aacf-9d1c71482808-kube-api-access-p7ncx\") pod \"b4854933-c4e8-437a-aacf-9d1c71482808\" (UID: \"b4854933-c4e8-437a-aacf-9d1c71482808\") "
Sep 11 19:31:00.156377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.156351    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4854933-c4e8-437a-aacf-9d1c71482808-kube-api-access-p7ncx" (OuterVolumeSpecName: "kube-api-access-p7ncx") pod "b4854933-c4e8-437a-aacf-9d1c71482808" (UID: "b4854933-c4e8-437a-aacf-9d1c71482808"). InnerVolumeSpecName "kube-api-access-p7ncx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:31:00.231199 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.231167    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:00.255115 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.255079    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7ncx\" (UniqueName: \"kubernetes.io/projected/b4854933-c4e8-437a-aacf-9d1c71482808-kube-api-access-p7ncx\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:31:00.331701 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.331654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:00.546465 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.546377    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4854933-c4e8-437a-aacf-9d1c71482808" path="/var/lib/kubelet/pods/b4854933-c4e8-437a-aacf-9d1c71482808/volumes"
Sep 11 19:31:00.806686 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.806599    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:00.941045 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.941012    2583 generic.go:356] "Generic (PLEG): container finished" podID="b4854933-c4e8-437a-aacf-9d1c71482808" containerID="913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e" exitCode=137
Sep 11 19:31:00.941492 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.941056    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7810-456f26"
Sep 11 19:31:00.941492 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.941111    2583 scope.go:117] "RemoveContainer" containerID="913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e"
Sep 11 19:31:00.949918 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.949899    2583 scope.go:117] "RemoveContainer" containerID="913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e"
Sep 11 19:31:00.950273 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:31:00.950231    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e\": container with ID starting with 913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e not found: ID does not exist" containerID="913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e"
Sep 11 19:31:00.950327 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:00.950283    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e"} err="failed to get container status \"913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e\": rpc error: code = NotFound desc = could not find container \"913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e\": container with ID starting with 913f1e821357abeae589d77edbfeda97ed9727dd61fd8f745130bea342f55d3e not found: ID does not exist"
Sep 11 19:31:01.075416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:01.075323    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:01.317064 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:01.317028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:01.969741 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:01.969708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:02.588335 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:02.588305    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:02.684783 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:02.684751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:02.786627 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:02.786600    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:03.085850 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:03.085816    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:03.186344 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:03.186308    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:03.643608 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:03.643574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:04.526407 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:04.526374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:04.941265 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:04.941208    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:31:04.941574 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:04.941554    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:31:04.941680 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:04.941640    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:31:05.001462 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:05.001395    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6" podStartSLOduration=8.001374501 podStartE2EDuration="8.001374501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:30:58.96153832 +0000 UTC m=+762.890163213" watchObservedRunningTime="2026-09-11 19:31:05.001374501 +0000 UTC m=+768.929999400"
Sep 11 19:31:05.828366 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:05.828332    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:05.929689 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:05.929642    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:06.223702 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.223664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:06.422514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.422464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:06.519304 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.519205    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:06.559445 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.559399    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"]
Sep 11 19:31:06.564197 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.564179    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:06.566654 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.566630    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-cac5b9bc\""
Sep 11 19:31:06.566771 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.566678    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-cac5b9bc-dockercfg-ztkmd\""
Sep 11 19:31:06.572564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.572543    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"]
Sep 11 19:31:06.616976 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.616945    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8993b0d-c053-460c-ab8c-518cb9420850-plugins-config-volume\") pod \"payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk\" (UID: \"a8993b0d-c053-460c-ab8c-518cb9420850\") " pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:06.617146 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.617110    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75zbv\" (UniqueName: \"kubernetes.io/projected/a8993b0d-c053-460c-ab8c-518cb9420850-kube-api-access-75zbv\") pod \"payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk\" (UID: \"a8993b0d-c053-460c-ab8c-518cb9420850\") " pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:06.717851 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.717813    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75zbv\" (UniqueName: \"kubernetes.io/projected/a8993b0d-c053-460c-ab8c-518cb9420850-kube-api-access-75zbv\") pod \"payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk\" (UID: \"a8993b0d-c053-460c-ab8c-518cb9420850\") " pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:06.718053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.717886    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8993b0d-c053-460c-ab8c-518cb9420850-plugins-config-volume\") pod \"payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk\" (UID: \"a8993b0d-c053-460c-ab8c-518cb9420850\") " pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:06.718600 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.718579    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8993b0d-c053-460c-ab8c-518cb9420850-plugins-config-volume\") pod \"payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk\" (UID: \"a8993b0d-c053-460c-ab8c-518cb9420850\") " pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:06.725921 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.725892    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75zbv\" (UniqueName: \"kubernetes.io/projected/a8993b0d-c053-460c-ab8c-518cb9420850-kube-api-access-75zbv\") pod \"payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk\" (UID: \"a8993b0d-c053-460c-ab8c-518cb9420850\") " pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:06.868643 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.868590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:06.875554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.875524    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:06.951380 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:06.951099    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:07.151530 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:07.151507    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"]
Sep 11 19:31:07.153584 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:31:07.153554    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8993b0d_c053_460c_ab8c_518cb9420850.slice/crio-3b3c267f63b912c7bbc996ec3b550d593c37bddb7a95fb6558bedd282a5048c2 WatchSource:0}: Error finding container 3b3c267f63b912c7bbc996ec3b550d593c37bddb7a95fb6558bedd282a5048c2: Status 404 returned error can't find the container with id 3b3c267f63b912c7bbc996ec3b550d593c37bddb7a95fb6558bedd282a5048c2
Sep 11 19:31:07.906918 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:07.906883    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:31:07.907434 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:07.906970    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:31:07.981623 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:07.981576    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk" event={"ID":"a8993b0d-c053-460c-ab8c-518cb9420850","Type":"ContainerStarted","Data":"35e76315e93eda3547d01132012bc93b9f67875a41bf0d3d6baa4bd56432710a"}
Sep 11 19:31:07.981826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:07.981657    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk" event={"ID":"a8993b0d-c053-460c-ab8c-518cb9420850","Type":"ContainerStarted","Data":"3b3c267f63b912c7bbc996ec3b550d593c37bddb7a95fb6558bedd282a5048c2"}
Sep 11 19:31:07.981826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:07.981689    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:07.998923 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:07.998874    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk" podStartSLOduration=1.9988583 podStartE2EDuration="1.9988583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:31:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:31:07.998070359 +0000 UTC m=+771.926695258" watchObservedRunningTime="2026-09-11 19:31:07.9988583 +0000 UTC m=+771.927483197"
Sep 11 19:31:08.062125 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:08.062086    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:08.752925 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:08.752887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:08.876766 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:08.876725    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:09.339572 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.339515    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:09.433881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.433844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:09.511826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.511793    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"]
Sep 11 19:31:09.515737 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.515711    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.518369 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.518346    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-6afc47-serving-cert\""
Sep 11 19:31:09.523091 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.523037    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:09.532723 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.532697    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"]
Sep 11 19:31:09.620183 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.620143    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:09.648210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.648178    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-metrics-certs\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.648430 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.648398    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-maas-api-tls\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.648508 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.648449    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpv7h\" (UniqueName: \"kubernetes.io/projected/5ad982c2-6fce-403a-a481-7cf8e70f334a-kube-api-access-zpv7h\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.648668 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.648578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad982c2-6fce-403a-a481-7cf8e70f334a-openshift-service-ca\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.728675 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.728619    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:09.749760 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.749726    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-metrics-certs\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.749930 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.749780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-maas-api-tls\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.749930 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.749864    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpv7h\" (UniqueName: \"kubernetes.io/projected/5ad982c2-6fce-403a-a481-7cf8e70f334a-kube-api-access-zpv7h\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.749930 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.749925    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad982c2-6fce-403a-a481-7cf8e70f334a-openshift-service-ca\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.750846 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.750820    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad982c2-6fce-403a-a481-7cf8e70f334a-openshift-service-ca\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.754360 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.752882    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-maas-api-tls\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.754475 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.754398    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-metrics-certs\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.760433 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.760412    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpv7h\" (UniqueName: \"kubernetes.io/projected/5ad982c2-6fce-403a-a481-7cf8e70f334a-kube-api-access-zpv7h\") pod \"maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.817811 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.817775    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:09.828699 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.828672    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:09.926666 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.917897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:09.941096 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:09.941065    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:31:10.109673 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:10.109646    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"]
Sep 11 19:31:10.113072 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:31:10.113038    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad982c2_6fce_403a_a481_7cf8e70f334a.slice/crio-4247e1e2a9c60c235bc4b10df84af099875f649073a9a4ba7a68dadb91b160d9 WatchSource:0}: Error finding container 4247e1e2a9c60c235bc4b10df84af099875f649073a9a4ba7a68dadb91b160d9: Status 404 returned error can't find the container with id 4247e1e2a9c60c235bc4b10df84af099875f649073a9a4ba7a68dadb91b160d9
Sep 11 19:31:10.166975 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:10.166878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:10.272213 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:10.272173    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:10.744366 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:10.744234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:11.000793 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:11.000763    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs" event={"ID":"5ad982c2-6fce-403a-a481-7cf8e70f334a","Type":"ContainerStarted","Data":"db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27"}
Sep 11 19:31:11.000995 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:11.000813    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs" event={"ID":"5ad982c2-6fce-403a-a481-7cf8e70f334a","Type":"ContainerStarted","Data":"4247e1e2a9c60c235bc4b10df84af099875f649073a9a4ba7a68dadb91b160d9"}
Sep 11 19:31:11.000995 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:11.000878    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:11.019765 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:11.019707    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs" podStartSLOduration=1.593709904 podStartE2EDuration="2.019683609s" totalImagesPullingTime="425.973705ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:31:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:31:11.018965522 +0000 UTC m=+774.947590422" watchObservedRunningTime="2026-09-11 19:31:11.019683609 +0000 UTC m=+774.948308507"
Sep 11 19:31:11.610771 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:11.610728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:12.521877 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:12.521824    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:12.620459 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:12.620424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:13.293183 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:13.293154    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:13.952018 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:13.951984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:14.053043 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:14.053009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:14.158619 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:14.158590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:14.291439 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:14.291334    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:14.457452 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:14.457421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:14.573280 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:14.573166    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:14.704747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:14.704715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:14.840742 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:14.840660    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:15.001197 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:15.001166    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:15.275513 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:15.275481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:15.876343 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:15.876308    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:15.974305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:15.974268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:16.100613 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:16.100581    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:16.350122 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:16.350092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:17.010495 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:17.010465    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:31:17.319786 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:17.319708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:17.831144 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:17.831098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:17.963167 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:17.963131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:18.336580 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:18.336545    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:18.988819 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:18.988784    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:19.125260 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:19.125212    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:19.225538 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:19.225505    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:19.723051 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:19.723015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:19.820198 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:19.820157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:19.853897 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:19.853860    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"]
Sep 11 19:31:19.858715 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:19.858693    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:31:19.862020 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:19.861997    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-515a4f-dockercfg-g579g\""
Sep 11 19:31:19.862161 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:19.862092    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-515a4f\""
Sep 11 19:31:19.870570 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:19.870546    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"]
Sep 11 19:31:19.951634 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:19.951601    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf66b\" (UniqueName: \"kubernetes.io/projected/a1894a02-4494-4cc0-970d-ba97f2309187-kube-api-access-mf66b\") pod \"payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt\" (UID: \"a1894a02-4494-4cc0-970d-ba97f2309187\") " pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:31:19.951800 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:19.951643    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1894a02-4494-4cc0-970d-ba97f2309187-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt\" (UID: \"a1894a02-4494-4cc0-970d-ba97f2309187\") " pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:31:20.053073 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:20.052975    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf66b\" (UniqueName: \"kubernetes.io/projected/a1894a02-4494-4cc0-970d-ba97f2309187-kube-api-access-mf66b\") pod \"payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt\" (UID: \"a1894a02-4494-4cc0-970d-ba97f2309187\") " pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:31:20.053073 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:20.053029    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1894a02-4494-4cc0-970d-ba97f2309187-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt\" (UID: \"a1894a02-4494-4cc0-970d-ba97f2309187\") " pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:31:20.053778 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:20.053757    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1894a02-4494-4cc0-970d-ba97f2309187-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt\" (UID: \"a1894a02-4494-4cc0-970d-ba97f2309187\") " pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:31:20.060968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:20.060934    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf66b\" (UniqueName: \"kubernetes.io/projected/a1894a02-4494-4cc0-970d-ba97f2309187-kube-api-access-mf66b\") pod \"payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt\" (UID: \"a1894a02-4494-4cc0-970d-ba97f2309187\") " pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:31:20.171162 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:20.171117    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:31:20.351209 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:20.351163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:20.444653 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:20.444631    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"]
Sep 11 19:31:20.446495 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:31:20.446461    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1894a02_4494_4cc0_970d_ba97f2309187.slice/crio-e72dd2f5a5896ce535b708f51323a3c47cf60ed21ec13aeb4cb84f100f9f96ad WatchSource:0}: Error finding container e72dd2f5a5896ce535b708f51323a3c47cf60ed21ec13aeb4cb84f100f9f96ad: Status 404 returned error can't find the container with id e72dd2f5a5896ce535b708f51323a3c47cf60ed21ec13aeb4cb84f100f9f96ad
Sep 11 19:31:21.029484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:21.029447    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:21.054986 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:21.054950    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt" event={"ID":"a1894a02-4494-4cc0-970d-ba97f2309187","Type":"ContainerStarted","Data":"396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b"}
Sep 11 19:31:21.055168 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:21.055008    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt" event={"ID":"a1894a02-4494-4cc0-970d-ba97f2309187","Type":"ContainerStarted","Data":"e72dd2f5a5896ce535b708f51323a3c47cf60ed21ec13aeb4cb84f100f9f96ad"}
Sep 11 19:31:21.055168 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:21.055045    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:31:21.075627 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:21.075578    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt" podStartSLOduration=2.075560327 podStartE2EDuration="2.075560327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:31:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:31:21.073374703 +0000 UTC m=+785.001999612" watchObservedRunningTime="2026-09-11 19:31:21.075560327 +0000 UTC m=+785.004185224"
Sep 11 19:31:21.920887 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:21.920851    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:22.310530 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:22.310453    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:22.432925 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:22.432889    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:22.557577 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:22.557545    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:22.679802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:22.679767    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:22.780376 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:22.780341    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:23.635953 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:23.635918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:24.409439 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:24.409409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:25.222843 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:25.222802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:25.884654 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:25.884616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:26.546212 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:26.546181    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:26.647062 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:26.647027    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:27.428159 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:27.428118    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:28.286975 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:28.286941    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:28.955167 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:28.955136    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:29.422129 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:29.422099    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:29.495390 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:29.495356    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:30.854062 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:30.854028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:32.062440 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:32.062406    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:31:33.087004 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:33.086969    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:33.852268 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:33.852214    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:33.954563 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:33.954525    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"]
Sep 11 19:31:33.955076 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:33.955025    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk" podUID="a8993b0d-c053-460c-ab8c-518cb9420850" containerName="payload-processing" containerID="cri-o://35e76315e93eda3547d01132012bc93b9f67875a41bf0d3d6baa4bd56432710a" gracePeriod=30
Sep 11 19:31:34.013675 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.013547    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:34.120377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.120301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:34.124361 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.124332    2583 generic.go:356] "Generic (PLEG): container finished" podID="a8993b0d-c053-460c-ab8c-518cb9420850" containerID="35e76315e93eda3547d01132012bc93b9f67875a41bf0d3d6baa4bd56432710a" exitCode=0
Sep 11 19:31:34.124500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.124410    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk" event={"ID":"a8993b0d-c053-460c-ab8c-518cb9420850","Type":"ContainerDied","Data":"35e76315e93eda3547d01132012bc93b9f67875a41bf0d3d6baa4bd56432710a"}
Sep 11 19:31:34.209037 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.209010    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:34.221415 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.221388    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:34.290512 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.290481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:34.294577 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.294553    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8993b0d-c053-460c-ab8c-518cb9420850-plugins-config-volume\") pod \"a8993b0d-c053-460c-ab8c-518cb9420850\" (UID: \"a8993b0d-c053-460c-ab8c-518cb9420850\") "
Sep 11 19:31:34.294714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.294618    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75zbv\" (UniqueName: \"kubernetes.io/projected/a8993b0d-c053-460c-ab8c-518cb9420850-kube-api-access-75zbv\") pod \"a8993b0d-c053-460c-ab8c-518cb9420850\" (UID: \"a8993b0d-c053-460c-ab8c-518cb9420850\") "
Sep 11 19:31:34.294970 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.294944    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8993b0d-c053-460c-ab8c-518cb9420850-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8993b0d-c053-460c-ab8c-518cb9420850" (UID: "a8993b0d-c053-460c-ab8c-518cb9420850"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:31:34.300419 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.300370    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8993b0d-c053-460c-ab8c-518cb9420850-kube-api-access-75zbv" (OuterVolumeSpecName: "kube-api-access-75zbv") pod "a8993b0d-c053-460c-ab8c-518cb9420850" (UID: "a8993b0d-c053-460c-ab8c-518cb9420850"). InnerVolumeSpecName "kube-api-access-75zbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:31:34.363155 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.363124    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:34.395809 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.395718    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8993b0d-c053-460c-ab8c-518cb9420850-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:31:34.395809 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.395749    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75zbv\" (UniqueName: \"kubernetes.io/projected/a8993b0d-c053-460c-ab8c-518cb9420850-kube-api-access-75zbv\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:31:34.737727 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.737635    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:34.875187 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.875155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:34.969200 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:34.969162    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:35.070291 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:35.070170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:35.130697 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:35.130663    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk" event={"ID":"a8993b0d-c053-460c-ab8c-518cb9420850","Type":"ContainerDied","Data":"3b3c267f63b912c7bbc996ec3b550d593c37bddb7a95fb6558bedd282a5048c2"}
Sep 11 19:31:35.131158 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:35.130722    2583 scope.go:117] "RemoveContainer" containerID="35e76315e93eda3547d01132012bc93b9f67875a41bf0d3d6baa4bd56432710a"
Sep 11 19:31:35.131158 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:35.130678    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"
Sep 11 19:31:35.150420 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:35.150383    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"]
Sep 11 19:31:35.156474 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:35.156445    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cac5b9bc-57bdff867-pwnsk"]
Sep 11 19:31:35.377461 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:35.377426    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:35.753362 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:35.753277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:35.857019 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:35.856982    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:36.546621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:36.546587    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8993b0d-c053-460c-ab8c-518cb9420850" path="/var/lib/kubelet/pods/a8993b0d-c053-460c-ab8c-518cb9420850/volumes"
Sep 11 19:31:36.656007 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:36.655971    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:37.336136 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:37.336097    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:37.669258 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:37.669208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:38.201454 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:38.201422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:38.363120 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:38.363080    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:38.463802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:38.463719    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:39.026443 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:39.026417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:39.941519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:39.941482    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:40.663145 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:40.663111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:41.458283 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:41.458228    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:41.557507 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:41.557477    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:41.658036 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:41.658005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:41.924288 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:41.924255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:42.017759 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.017724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:42.060395 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.060361    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"]
Sep 11 19:31:42.060877 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.060850    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8993b0d-c053-460c-ab8c-518cb9420850" containerName="payload-processing"
Sep 11 19:31:42.060877 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.060870    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8993b0d-c053-460c-ab8c-518cb9420850" containerName="payload-processing"
Sep 11 19:31:42.061026 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.060959    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8993b0d-c053-460c-ab8c-518cb9420850" containerName="payload-processing"
Sep 11 19:31:42.064012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.063996    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:31:42.066424 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.066400    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-da3756c4-dockercfg-vdw6z\""
Sep 11 19:31:42.066537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.066406    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-da3756c4\""
Sep 11 19:31:42.076334 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.076311    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"]
Sep 11 19:31:42.170411 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.170373    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18085881-b07f-45d2-8bf4-4a0731441da5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l\" (UID: \"18085881-b07f-45d2-8bf4-4a0731441da5\") " pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:31:42.170607 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.170418    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7jtp\" (UniqueName: \"kubernetes.io/projected/18085881-b07f-45d2-8bf4-4a0731441da5-kube-api-access-s7jtp\") pod \"payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l\" (UID: \"18085881-b07f-45d2-8bf4-4a0731441da5\") " pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:31:42.271689 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.271608    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18085881-b07f-45d2-8bf4-4a0731441da5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l\" (UID: \"18085881-b07f-45d2-8bf4-4a0731441da5\") " pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:31:42.271689 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.271648    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7jtp\" (UniqueName: \"kubernetes.io/projected/18085881-b07f-45d2-8bf4-4a0731441da5-kube-api-access-s7jtp\") pod \"payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l\" (UID: \"18085881-b07f-45d2-8bf4-4a0731441da5\") " pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:31:42.272333 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.272314    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18085881-b07f-45d2-8bf4-4a0731441da5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l\" (UID: \"18085881-b07f-45d2-8bf4-4a0731441da5\") " pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:31:42.279167 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.279139    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7jtp\" (UniqueName: \"kubernetes.io/projected/18085881-b07f-45d2-8bf4-4a0731441da5-kube-api-access-s7jtp\") pod \"payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l\" (UID: \"18085881-b07f-45d2-8bf4-4a0731441da5\") " pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:31:42.331538 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.331509    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:42.374320 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.374296    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:31:42.646544 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:42.646513    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"]
Sep 11 19:31:42.651090 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:31:42.651061    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18085881_b07f_45d2_8bf4_4a0731441da5.slice/crio-cbf462a77c6276d56d8dc508e98a8ad8c300f56ec32ed7dd766bf478c4434807 WatchSource:0}: Error finding container cbf462a77c6276d56d8dc508e98a8ad8c300f56ec32ed7dd766bf478c4434807: Status 404 returned error can't find the container with id cbf462a77c6276d56d8dc508e98a8ad8c300f56ec32ed7dd766bf478c4434807
Sep 11 19:31:43.173134 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:43.173101    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l" event={"ID":"18085881-b07f-45d2-8bf4-4a0731441da5","Type":"ContainerStarted","Data":"9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295"}
Sep 11 19:31:43.173563 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:43.173154    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l" event={"ID":"18085881-b07f-45d2-8bf4-4a0731441da5","Type":"ContainerStarted","Data":"cbf462a77c6276d56d8dc508e98a8ad8c300f56ec32ed7dd766bf478c4434807"}
Sep 11 19:31:43.173563 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:43.173178    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:31:43.192066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:43.192014    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l" podStartSLOduration=1.191997887 podStartE2EDuration="1.191997887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:31:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:31:43.189920335 +0000 UTC m=+807.118545269" watchObservedRunningTime="2026-09-11 19:31:43.191997887 +0000 UTC m=+807.120622785"
Sep 11 19:31:43.224437 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:43.224409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:43.897900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:43.897857    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:44.576875 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:44.576844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:31:54.181711 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:31:54.181677    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:32:03.727912 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:03.727879    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:04.556009 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:04.555976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:05.434852 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:05.434819    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:06.149071 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:06.149041    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:06.879216 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:06.879175    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:07.841487 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:07.841448    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:08.600844 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:08.600794    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:09.262877 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:09.262833    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:09.901100 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:09.901068    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:09.928016 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:09.927983    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"]
Sep 11 19:32:09.928282 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:09.928235    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l" podUID="18085881-b07f-45d2-8bf4-4a0731441da5" containerName="payload-processing" containerID="cri-o://9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295" gracePeriod=30
Sep 11 19:32:10.014154 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.014119    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:10.105001 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.104970    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:10.183307 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.183281    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:32:10.230934 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.230904    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18085881-b07f-45d2-8bf4-4a0731441da5-plugins-config-volume\") pod \"18085881-b07f-45d2-8bf4-4a0731441da5\" (UID: \"18085881-b07f-45d2-8bf4-4a0731441da5\") "
Sep 11 19:32:10.231312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.231290    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18085881-b07f-45d2-8bf4-4a0731441da5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "18085881-b07f-45d2-8bf4-4a0731441da5" (UID: "18085881-b07f-45d2-8bf4-4a0731441da5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:32:10.236814 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.236786    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:10.308944 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.308906    2583 generic.go:356] "Generic (PLEG): container finished" podID="18085881-b07f-45d2-8bf4-4a0731441da5" containerID="9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295" exitCode=0
Sep 11 19:32:10.309110 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.308956    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"
Sep 11 19:32:10.309110 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.308990    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l" event={"ID":"18085881-b07f-45d2-8bf4-4a0731441da5","Type":"ContainerDied","Data":"9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295"}
Sep 11 19:32:10.309110 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.309046    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l" event={"ID":"18085881-b07f-45d2-8bf4-4a0731441da5","Type":"ContainerDied","Data":"cbf462a77c6276d56d8dc508e98a8ad8c300f56ec32ed7dd766bf478c4434807"}
Sep 11 19:32:10.309110 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.309060    2583 scope.go:117] "RemoveContainer" containerID="9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295"
Sep 11 19:32:10.320509 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.320478    2583 scope.go:117] "RemoveContainer" containerID="9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295"
Sep 11 19:32:10.320889 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:32:10.320859    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295\": container with ID starting with 9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295 not found: ID does not exist" containerID="9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295"
Sep 11 19:32:10.320981 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.320901    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295"} err="failed to get container status \"9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295\": rpc error: code = NotFound desc = could not find container \"9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295\": container with ID starting with 9c23341314cda298e9415bb309d07e6259c033738c4ef2fce1916604ac89b295 not found: ID does not exist"
Sep 11 19:32:10.332264 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.332226    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7jtp\" (UniqueName: \"kubernetes.io/projected/18085881-b07f-45d2-8bf4-4a0731441da5-kube-api-access-s7jtp\") pod \"18085881-b07f-45d2-8bf4-4a0731441da5\" (UID: \"18085881-b07f-45d2-8bf4-4a0731441da5\") "
Sep 11 19:32:10.332489 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.332477    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18085881-b07f-45d2-8bf4-4a0731441da5-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:32:10.334428 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.334401    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18085881-b07f-45d2-8bf4-4a0731441da5-kube-api-access-s7jtp" (OuterVolumeSpecName: "kube-api-access-s7jtp") pod "18085881-b07f-45d2-8bf4-4a0731441da5" (UID: "18085881-b07f-45d2-8bf4-4a0731441da5"). InnerVolumeSpecName "kube-api-access-s7jtp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:32:10.433219 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.433128    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7jtp\" (UniqueName: \"kubernetes.io/projected/18085881-b07f-45d2-8bf4-4a0731441da5-kube-api-access-s7jtp\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:32:10.539545 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.539485    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:10.634150 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.634119    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"]
Sep 11 19:32:10.651954 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.651921    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-da3756c4-56dcf9c998-cp62l"]
Sep 11 19:32:10.759015 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.758926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:10.857866 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:10.857834    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:11.216559 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:11.216529    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:11.375523 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:11.375489    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:11.726534 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:11.726481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:12.444988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:12.444952    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:12.546829 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:12.546797    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18085881-b07f-45d2-8bf4-4a0731441da5" path="/var/lib/kubelet/pods/18085881-b07f-45d2-8bf4-4a0731441da5/volumes"
Sep 11 19:32:13.311576 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:13.311538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:13.726714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:13.726678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:13.823230 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:13.823185    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:14.505542 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:14.505507    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:15.259818 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:15.259769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:16.152649 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:16.152614    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:16.253517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:16.253487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:16.495811 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:16.495736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:16.746798 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:16.746718    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:16.845806 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:16.845776    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:17.139951 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:17.139908    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:18.120235 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:18.120199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:18.838087 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:18.838053    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:19.529889 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:19.529855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:28.167522 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:28.167485    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:38.209741 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:38.209702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:39.055104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:39.055058    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:39.844716 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:39.844685    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:40.785553 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:40.785518    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:41.448822 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:41.448769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:42.135511 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:42.135467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:42.944787 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:42.944753    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:43.705110 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:43.705074    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:44.618919 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:44.618889    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:45.249230 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:45.249199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:45.315667 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:45.315637    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:45.379140 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:45.379110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:45.444034 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:45.444005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:45.522756 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:45.522677    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:45.840199 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:45.840104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:46.557267 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:46.557216    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:46.660302 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:46.660268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:47.080533 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:47.080498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:47.357384 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:47.357354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:47.458322 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:47.458275    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:47.938124 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:47.938089    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:48.236183 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:48.236075    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:49.059631 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.059582    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:49.299928 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.299890    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"]
Sep 11 19:32:49.300341 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.300323    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18085881-b07f-45d2-8bf4-4a0731441da5" containerName="payload-processing"
Sep 11 19:32:49.300397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.300344    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="18085881-b07f-45d2-8bf4-4a0731441da5" containerName="payload-processing"
Sep 11 19:32:49.300438 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.300428    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="18085881-b07f-45d2-8bf4-4a0731441da5" containerName="payload-processing"
Sep 11 19:32:49.303337 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.303322    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.305904 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.305883    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-1fc808\"/\"openshift-service-ca.crt\""
Sep 11 19:32:49.306640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.306622    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1fc808\"/\"simulator-w0-1fc808-kserve-self-signed-certs\""
Sep 11 19:32:49.306640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.306636    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1fc808\"/\"default-dockercfg-8gddw\""
Sep 11 19:32:49.306806 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.306643    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-1fc808\"/\"kube-root-ca.crt\""
Sep 11 19:32:49.311491 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.311470    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"]
Sep 11 19:32:49.398851 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.398823    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-tmp-dir\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.398851 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.398858    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-model-cache\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.399055 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.398881    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-home\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.399055 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.398934    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-dshm\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.399055 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.399043    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57932cd5-9b5f-48e4-b747-572df168c661-tls-certs\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.399155 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.399112    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff88t\" (UniqueName: \"kubernetes.io/projected/57932cd5-9b5f-48e4-b747-572df168c661-kube-api-access-ff88t\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.405825 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.405785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:49.499657 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.499620    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57932cd5-9b5f-48e4-b747-572df168c661-tls-certs\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.499865 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.499791    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff88t\" (UniqueName: \"kubernetes.io/projected/57932cd5-9b5f-48e4-b747-572df168c661-kube-api-access-ff88t\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.499937 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.499870    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-tmp-dir\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.499937 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.499911    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-model-cache\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.500034 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.499945    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-home\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.500034 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.499970    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-dshm\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.500324 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.500301    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-tmp-dir\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.500460 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.500438    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-home\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.500525 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.500424    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-model-cache\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.502398 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.502375    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-dshm\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.502853 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.502834    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57932cd5-9b5f-48e4-b747-572df168c661-tls-certs\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.508095 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.508072    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff88t\" (UniqueName: \"kubernetes.io/projected/57932cd5-9b5f-48e4-b747-572df168c661-kube-api-access-ff88t\") pod \"simulator-w0-1fc808-kserve-74556bbcf7-9zdw7\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") " pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.613825 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.613790    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:49.830266 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.830216    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:49.894837 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.894810    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"]
Sep 11 19:32:49.896648 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:32:49.896621    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57932cd5_9b5f_48e4_b747_572df168c661.slice/crio-c1ab690454979eb9cd57307b755741685f230ece965de7f72a7eae669030218d WatchSource:0}: Error finding container c1ab690454979eb9cd57307b755741685f230ece965de7f72a7eae669030218d: Status 404 returned error can't find the container with id c1ab690454979eb9cd57307b755741685f230ece965de7f72a7eae669030218d
Sep 11 19:32:49.901222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:49.901201    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:50.512438 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:50.512406    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7" event={"ID":"57932cd5-9b5f-48e4-b747-572df168c661","Type":"ContainerStarted","Data":"6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e"}
Sep 11 19:32:50.512865 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:50.512450    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7" event={"ID":"57932cd5-9b5f-48e4-b747-572df168c661","Type":"ContainerStarted","Data":"c1ab690454979eb9cd57307b755741685f230ece965de7f72a7eae669030218d"}
Sep 11 19:32:50.534125 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:50.534068    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7" podStartSLOduration=1.534050191 podStartE2EDuration="1.534050191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:32:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:32:50.53248819 +0000 UTC m=+874.461113090" watchObservedRunningTime="2026-09-11 19:32:50.534050191 +0000 UTC m=+874.462675089"
Sep 11 19:32:50.868681 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:50.868621    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:51.781236 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:51.781205    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:52.636201 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:52.636161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:53.432989 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:53.432956    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:54.140950 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:54.140921    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:54.241305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:54.241271    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:54.450897 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:54.450802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:54.634101 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:54.634069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:54.733310 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:54.733194    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:32:59.614290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:59.614253    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:59.614935 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:59.614311    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:32:59.628072 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:32:59.628040    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:33:00.579435 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:00.579410    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:33:16.072182 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:16.072091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:16.942527 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:16.942490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:17.630438 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:17.630405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:18.294934 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:18.294903    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:19.009587 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:19.009552    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:19.705472 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:19.705438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:20.392038 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:20.392004    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:21.157089 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:21.157056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:21.831272 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:21.831215    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:22.398923 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.398887    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"]
Sep 11 19:33:22.402859 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.402836    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.405068 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.405039    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-8fbb58\"/\"openshift-service-ca.crt\""
Sep 11 19:33:22.405208 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.405161    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-8fbb58\"/\"kube-root-ca.crt\""
Sep 11 19:33:22.405969 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.405952    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-8fbb58\"/\"simulator-w4-8fbb58-kserve-self-signed-certs\""
Sep 11 19:33:22.406067 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.405988    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-8fbb58\"/\"default-dockercfg-vx2j9\""
Sep 11 19:33:22.413554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.413533    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"]
Sep 11 19:33:22.506372 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.506337    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:22.521345 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.521312    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tmp-dir\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.521542 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.521363    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9kqh\" (UniqueName: \"kubernetes.io/projected/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-kube-api-access-s9kqh\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.521542 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.521386    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-dshm\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.521542 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.521465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-model-cache\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.521542 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.521514    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-home\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.521542 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.521534    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tls-certs\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.567031 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.567001    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:22.623037 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.622997    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-model-cache\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.623230 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.623054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-home\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.623230 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.623076    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tls-certs\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.623230 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.623143    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tmp-dir\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.623230 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.623174    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9kqh\" (UniqueName: \"kubernetes.io/projected/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-kube-api-access-s9kqh\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.623505 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.623447    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-dshm\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.623505 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.623476    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-model-cache\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.623610 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.623492    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tmp-dir\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.623664 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.623644    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-home\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.625661 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.625631    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-dshm\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.625816 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.625800    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tls-certs\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.631372 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.631348    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9kqh\" (UniqueName: \"kubernetes.io/projected/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-kube-api-access-s9kqh\") pod \"simulator-w4-8fbb58-kserve-57646df67b-6jxds\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") " pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.713558 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.713471    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:22.985418 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:22.985392    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"]
Sep 11 19:33:22.987200 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:33:22.987171    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f8e5763_e2fe_42b7_a52b_6f8d901ce644.slice/crio-5e12ca4b9a06c26e905827010d6dbef1814794c79c5eadde2dc898f3b71028c0 WatchSource:0}: Error finding container 5e12ca4b9a06c26e905827010d6dbef1814794c79c5eadde2dc898f3b71028c0: Status 404 returned error can't find the container with id 5e12ca4b9a06c26e905827010d6dbef1814794c79c5eadde2dc898f3b71028c0
Sep 11 19:33:23.237582 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:23.237494    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:23.684391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:23.684354    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds" event={"ID":"7f8e5763-e2fe-42b7-a52b-6f8d901ce644","Type":"ContainerStarted","Data":"9fcb879d1bacaf9a0773c5c3383370390eabe1b657fbe78990b5537b7b5598ba"}
Sep 11 19:33:23.684391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:23.684395    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds" event={"ID":"7f8e5763-e2fe-42b7-a52b-6f8d901ce644","Type":"ContainerStarted","Data":"5e12ca4b9a06c26e905827010d6dbef1814794c79c5eadde2dc898f3b71028c0"}
Sep 11 19:33:24.040178 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:24.040092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:24.871915 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:24.871876    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:29.105379 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:29.105345    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:31.159885 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:31.159852    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:31.772959 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:31.772923    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:31.840583 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:31.840553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:31.940866 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:31.940826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:31.998726 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:31.998694    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:32.114594 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:32.114556    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:32.415128 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:32.415046    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:32.713743 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:32.713652    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:32.713743 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:32.713706    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:32.726560 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:32.726533    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:32.743282 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:32.743233    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:33:32.752847 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:32.752795    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds" podStartSLOduration=10.752778537 podStartE2EDuration="10.752778537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:33:23.720473606 +0000 UTC m=+907.649098504" watchObservedRunningTime="2026-09-11 19:33:32.752778537 +0000 UTC m=+916.681403435"
Sep 11 19:33:32.760146 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:32.760118    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:32.835053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:32.834943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:32.935456 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:32.935420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:33.297044 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:33.297015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:34.143188 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:34.143147    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:34.311112 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:34.311052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:35.008427 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:35.008387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:35.738540 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:35.738507    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:36.442444 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:36.442411    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:40.340937 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:40.340897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:40.402274 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:40.402210    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:40.493097 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:40.493063    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:40.598303 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:40.598277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:40.732298 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:40.732266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:41.027134 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:41.027043    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:33:44.300885 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:33:44.300842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:01.421008 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:01.420973    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:02.872763 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:02.872731    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:03.528260 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:03.528208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:04.339501 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:04.339474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:05.027712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:05.027650    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:05.681129 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:05.681092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:06.933974 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:06.933939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:11.864742 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:11.864708    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"]
Sep 11 19:34:11.865204 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:11.865003    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs" podUID="5ad982c2-6fce-403a-a481-7cf8e70f334a" containerName="maas-api" containerID="cri-o://db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27" gracePeriod=30
Sep 11 19:34:11.890553 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:11.890520    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:11.962507 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:11.962468    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:12.006790 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.006699    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs" podUID="5ad982c2-6fce-403a-a481-7cf8e70f334a" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.41:8443/health\": dial tcp 10.133.0.41:8443: connect: connection refused"
Sep 11 19:34:12.033332 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.033298    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:12.125472 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.125403    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:34:12.130583 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.130553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:12.199364 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.199328    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-maas-api-tls\") pod \"5ad982c2-6fce-403a-a481-7cf8e70f334a\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") "
Sep 11 19:34:12.199556 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.199416    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad982c2-6fce-403a-a481-7cf8e70f334a-openshift-service-ca\") pod \"5ad982c2-6fce-403a-a481-7cf8e70f334a\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") "
Sep 11 19:34:12.199556 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.199452    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-metrics-certs\") pod \"5ad982c2-6fce-403a-a481-7cf8e70f334a\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") "
Sep 11 19:34:12.199556 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.199495    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpv7h\" (UniqueName: \"kubernetes.io/projected/5ad982c2-6fce-403a-a481-7cf8e70f334a-kube-api-access-zpv7h\") pod \"5ad982c2-6fce-403a-a481-7cf8e70f334a\" (UID: \"5ad982c2-6fce-403a-a481-7cf8e70f334a\") "
Sep 11 19:34:12.199862 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.199817    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ad982c2-6fce-403a-a481-7cf8e70f334a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5ad982c2-6fce-403a-a481-7cf8e70f334a" (UID: "5ad982c2-6fce-403a-a481-7cf8e70f334a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:34:12.201964 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.201938    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5ad982c2-6fce-403a-a481-7cf8e70f334a" (UID: "5ad982c2-6fce-403a-a481-7cf8e70f334a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:34:12.202127 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.202110    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5ad982c2-6fce-403a-a481-7cf8e70f334a" (UID: "5ad982c2-6fce-403a-a481-7cf8e70f334a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:34:12.202204 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.202178    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ad982c2-6fce-403a-a481-7cf8e70f334a-kube-api-access-zpv7h" (OuterVolumeSpecName: "kube-api-access-zpv7h") pod "5ad982c2-6fce-403a-a481-7cf8e70f334a" (UID: "5ad982c2-6fce-403a-a481-7cf8e70f334a"). InnerVolumeSpecName "kube-api-access-zpv7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:34:12.300455 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.300416    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:12.300455 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.300448    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpv7h\" (UniqueName: \"kubernetes.io/projected/5ad982c2-6fce-403a-a481-7cf8e70f334a-kube-api-access-zpv7h\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:12.300455 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.300458    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad982c2-6fce-403a-a481-7cf8e70f334a-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:12.300455 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.300467    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad982c2-6fce-403a-a481-7cf8e70f334a-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:12.414996 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.414909    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:12.517504 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.517473    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:12.931033 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.930992    2583 generic.go:356] "Generic (PLEG): container finished" podID="5ad982c2-6fce-403a-a481-7cf8e70f334a" containerID="db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27" exitCode=0
Sep 11 19:34:12.931475 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.931045    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"
Sep 11 19:34:12.931475 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.931045    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs" event={"ID":"5ad982c2-6fce-403a-a481-7cf8e70f334a","Type":"ContainerDied","Data":"db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27"}
Sep 11 19:34:12.931475 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.931175    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs" event={"ID":"5ad982c2-6fce-403a-a481-7cf8e70f334a","Type":"ContainerDied","Data":"4247e1e2a9c60c235bc4b10df84af099875f649073a9a4ba7a68dadb91b160d9"}
Sep 11 19:34:12.931475 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.931204    2583 scope.go:117] "RemoveContainer" containerID="db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27"
Sep 11 19:34:12.940546 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.940527    2583 scope.go:117] "RemoveContainer" containerID="db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27"
Sep 11 19:34:12.940829 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:34:12.940812    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27\": container with ID starting with db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27 not found: ID does not exist" containerID="db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27"
Sep 11 19:34:12.940877 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.940837    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27"} err="failed to get container status \"db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27\": rpc error: code = NotFound desc = could not find container \"db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27\": container with ID starting with db2618a7cfc5819933d161436f771cd9c53f68fb3d37c217e9e7a392b22c6a27 not found: ID does not exist"
Sep 11 19:34:12.946334 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.946310    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"]
Sep 11 19:34:12.949872 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:12.949846    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-6afc47-57564bdbd8-pjcbs"]
Sep 11 19:34:13.192747 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:13.192655    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:13.624633 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:13.624598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:13.726670 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:13.726636    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:13.832550 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:13.832514    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:14.051122 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:14.051033    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:14.546520 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:14.546488    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ad982c2-6fce-403a-a481-7cf8e70f334a" path="/var/lib/kubelet/pods/5ad982c2-6fce-403a-a481-7cf8e70f334a/volumes"
Sep 11 19:34:14.865437 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:14.865404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:15.670798 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:15.670769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:17.440022 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:17.439986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:19.925432 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:19.925396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:21.013155 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:21.013112    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:21.680829 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:21.680800    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:22.357399 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:22.357359    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:23.145302 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:23.145226    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:23.942554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:23.942498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:24.908253 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:24.908219    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"]
Sep 11 19:34:25.003617 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:25.003588    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:27.210705 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:27.210669    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:29.642535 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:29.642503    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:30.527309 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.527276    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"]
Sep 11 19:34:30.528130 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.528095    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd" podUID="26c66035-04c3-4037-b693-88ff86d1f870" containerName="maas-api" containerID="cri-o://aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3" gracePeriod=30
Sep 11 19:34:30.559603 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.559570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:30.596029 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.595992    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"]
Sep 11 19:34:30.596340 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.596288    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6" podUID="78d4b411-e77d-4693-8aff-fc2d20dfd9c5" containerName="payload-processing" containerID="cri-o://577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f" gracePeriod=30
Sep 11 19:34:30.635172 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.635142    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:30.706627 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.706594    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:30.778257 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.778077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:30.843166 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.843137    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:34:30.846277 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.846234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:30.875001 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.874978    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:34:30.978974 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.978926    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-plugins-config-volume\") pod \"78d4b411-e77d-4693-8aff-fc2d20dfd9c5\" (UID: \"78d4b411-e77d-4693-8aff-fc2d20dfd9c5\") "
Sep 11 19:34:30.978974 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.978980    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dwh5\" (UniqueName: \"kubernetes.io/projected/26c66035-04c3-4037-b693-88ff86d1f870-kube-api-access-8dwh5\") pod \"26c66035-04c3-4037-b693-88ff86d1f870\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") "
Sep 11 19:34:30.979222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.979013    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-maas-api-tls\") pod \"26c66035-04c3-4037-b693-88ff86d1f870\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") "
Sep 11 19:34:30.979222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.979096    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26c66035-04c3-4037-b693-88ff86d1f870-openshift-service-ca\") pod \"26c66035-04c3-4037-b693-88ff86d1f870\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") "
Sep 11 19:34:30.979222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.979133    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-metrics-certs\") pod \"26c66035-04c3-4037-b693-88ff86d1f870\" (UID: \"26c66035-04c3-4037-b693-88ff86d1f870\") "
Sep 11 19:34:30.979222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.979181    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc6dc\" (UniqueName: \"kubernetes.io/projected/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-kube-api-access-nc6dc\") pod \"78d4b411-e77d-4693-8aff-fc2d20dfd9c5\" (UID: \"78d4b411-e77d-4693-8aff-fc2d20dfd9c5\") "
Sep 11 19:34:30.979529 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.979409    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "78d4b411-e77d-4693-8aff-fc2d20dfd9c5" (UID: "78d4b411-e77d-4693-8aff-fc2d20dfd9c5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:34:30.979587 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.979550    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:30.979782 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.979756    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26c66035-04c3-4037-b693-88ff86d1f870-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "26c66035-04c3-4037-b693-88ff86d1f870" (UID: "26c66035-04c3-4037-b693-88ff86d1f870"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:34:30.981870 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.981847    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "26c66035-04c3-4037-b693-88ff86d1f870" (UID: "26c66035-04c3-4037-b693-88ff86d1f870"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:34:30.982007 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.981964    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "26c66035-04c3-4037-b693-88ff86d1f870" (UID: "26c66035-04c3-4037-b693-88ff86d1f870"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:34:30.982007 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.981981    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26c66035-04c3-4037-b693-88ff86d1f870-kube-api-access-8dwh5" (OuterVolumeSpecName: "kube-api-access-8dwh5") pod "26c66035-04c3-4037-b693-88ff86d1f870" (UID: "26c66035-04c3-4037-b693-88ff86d1f870"). InnerVolumeSpecName "kube-api-access-8dwh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:34:30.982128 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:30.982008    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-kube-api-access-nc6dc" (OuterVolumeSpecName: "kube-api-access-nc6dc") pod "78d4b411-e77d-4693-8aff-fc2d20dfd9c5" (UID: "78d4b411-e77d-4693-8aff-fc2d20dfd9c5"). InnerVolumeSpecName "kube-api-access-nc6dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:34:31.022193 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.022160    2583 generic.go:356] "Generic (PLEG): container finished" podID="26c66035-04c3-4037-b693-88ff86d1f870" containerID="aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3" exitCode=0
Sep 11 19:34:31.022391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.022204    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"
Sep 11 19:34:31.022391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.022273    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd" event={"ID":"26c66035-04c3-4037-b693-88ff86d1f870","Type":"ContainerDied","Data":"aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3"}
Sep 11 19:34:31.022391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.022326    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd" event={"ID":"26c66035-04c3-4037-b693-88ff86d1f870","Type":"ContainerDied","Data":"7e458b9908de7ff835d253f7103ab6c6110f600677bed9c42c9853602a7ac626"}
Sep 11 19:34:31.022391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.022350    2583 scope.go:117] "RemoveContainer" containerID="aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3"
Sep 11 19:34:31.023796 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.023770    2583 generic.go:356] "Generic (PLEG): container finished" podID="78d4b411-e77d-4693-8aff-fc2d20dfd9c5" containerID="577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f" exitCode=0
Sep 11 19:34:31.023915 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.023820    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"
Sep 11 19:34:31.023915 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.023841    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6" event={"ID":"78d4b411-e77d-4693-8aff-fc2d20dfd9c5","Type":"ContainerDied","Data":"577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f"}
Sep 11 19:34:31.023915 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.023882    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6" event={"ID":"78d4b411-e77d-4693-8aff-fc2d20dfd9c5","Type":"ContainerDied","Data":"3deafdea69c3a843efb00adce55c9a876aa0b67ac1c085c9cbfa60a97a683aec"}
Sep 11 19:34:31.032651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.032634    2583 scope.go:117] "RemoveContainer" containerID="aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3"
Sep 11 19:34:31.032978 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:34:31.032951    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3\": container with ID starting with aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3 not found: ID does not exist" containerID="aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3"
Sep 11 19:34:31.033089 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.033002    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3"} err="failed to get container status \"aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3\": rpc error: code = NotFound desc = could not find container \"aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3\": container with ID starting with aebe1d91d58c9380db8fbf84fa93c04d65b2d907d92afbe3c1df469f813949b3 not found: ID does not exist"
Sep 11 19:34:31.033089 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.033028    2583 scope.go:117] "RemoveContainer" containerID="577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f"
Sep 11 19:34:31.044980 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.044961    2583 scope.go:117] "RemoveContainer" containerID="577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f"
Sep 11 19:34:31.045326 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:34:31.045306    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f\": container with ID starting with 577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f not found: ID does not exist" containerID="577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f"
Sep 11 19:34:31.045397 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.045338    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f"} err="failed to get container status \"577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f\": rpc error: code = NotFound desc = could not find container \"577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f\": container with ID starting with 577fe75322221134ac2284b313f4f0db81a365690f66c17e3ffbac9305d4f14f not found: ID does not exist"
Sep 11 19:34:31.045815 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.045798    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"]
Sep 11 19:34:31.052621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.052598    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-8fbb58-c495f7dd9-g4rfd"]
Sep 11 19:34:31.065006 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.064979    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"]
Sep 11 19:34:31.070226 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.070199    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8fbb58-65987b96df-ffwf6"]
Sep 11 19:34:31.080877 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.080853    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dwh5\" (UniqueName: \"kubernetes.io/projected/26c66035-04c3-4037-b693-88ff86d1f870-kube-api-access-8dwh5\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:31.080877 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.080879    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:31.081033 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.080889    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26c66035-04c3-4037-b693-88ff86d1f870-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:31.081033 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.080900    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26c66035-04c3-4037-b693-88ff86d1f870-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:31.081033 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.080909    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc6dc\" (UniqueName: \"kubernetes.io/projected/78d4b411-e77d-4693-8aff-fc2d20dfd9c5-kube-api-access-nc6dc\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:31.149599 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.149560    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:31.255146 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.255109    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:31.975319 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:31.975283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:32.546326 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:32.546293    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26c66035-04c3-4037-b693-88ff86d1f870" path="/var/lib/kubelet/pods/26c66035-04c3-4037-b693-88ff86d1f870/volumes"
Sep 11 19:34:32.546674 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:32.546661    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78d4b411-e77d-4693-8aff-fc2d20dfd9c5" path="/var/lib/kubelet/pods/78d4b411-e77d-4693-8aff-fc2d20dfd9c5/volumes"
Sep 11 19:34:32.837992 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:32.837908    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:33.636807 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:33.636772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:35.461972 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:35.461940    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:35.530587 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:35.530553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:35.612449 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:35.612419    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:36.673359 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:36.673307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:37.169423 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:37.169386    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:37.231324 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:37.231287    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:37.410846 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:37.410810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:39.928106 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:39.928059    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds" podUID="7f8e5763-e2fe-42b7-a52b-6f8d901ce644" containerName="main" containerID="cri-o://9fcb879d1bacaf9a0773c5c3383370390eabe1b657fbe78990b5537b7b5598ba" gracePeriod=45
Sep 11 19:34:40.080117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.080069    2583 generic.go:356] "Generic (PLEG): container finished" podID="7f8e5763-e2fe-42b7-a52b-6f8d901ce644" containerID="9fcb879d1bacaf9a0773c5c3383370390eabe1b657fbe78990b5537b7b5598ba" exitCode=0
Sep 11 19:34:40.080301 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.080133    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds" event={"ID":"7f8e5763-e2fe-42b7-a52b-6f8d901ce644","Type":"ContainerDied","Data":"9fcb879d1bacaf9a0773c5c3383370390eabe1b657fbe78990b5537b7b5598ba"}
Sep 11 19:34:40.176813 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.176789    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:34:40.271450 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.271354    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tls-certs\") pod \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") "
Sep 11 19:34:40.271450 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.271407    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-dshm\") pod \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") "
Sep 11 19:34:40.271669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.271455    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tmp-dir\") pod \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") "
Sep 11 19:34:40.271669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.271518    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9kqh\" (UniqueName: \"kubernetes.io/projected/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-kube-api-access-s9kqh\") pod \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") "
Sep 11 19:34:40.271669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.271550    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-model-cache\") pod \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") "
Sep 11 19:34:40.271669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.271606    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-home\") pod \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\" (UID: \"7f8e5763-e2fe-42b7-a52b-6f8d901ce644\") "
Sep 11 19:34:40.271860 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.271777    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7f8e5763-e2fe-42b7-a52b-6f8d901ce644" (UID: "7f8e5763-e2fe-42b7-a52b-6f8d901ce644"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:34:40.271918 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.271886    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-model-cache" (OuterVolumeSpecName: "model-cache") pod "7f8e5763-e2fe-42b7-a52b-6f8d901ce644" (UID: "7f8e5763-e2fe-42b7-a52b-6f8d901ce644"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:34:40.271977 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.271924    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-home" (OuterVolumeSpecName: "home") pod "7f8e5763-e2fe-42b7-a52b-6f8d901ce644" (UID: "7f8e5763-e2fe-42b7-a52b-6f8d901ce644"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:34:40.272020 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.271994    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tmp-dir\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:40.272020 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.272010    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-model-cache\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:40.273859 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.273837    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7f8e5763-e2fe-42b7-a52b-6f8d901ce644" (UID: "7f8e5763-e2fe-42b7-a52b-6f8d901ce644"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:34:40.273978 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.273874    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-dshm" (OuterVolumeSpecName: "dshm") pod "7f8e5763-e2fe-42b7-a52b-6f8d901ce644" (UID: "7f8e5763-e2fe-42b7-a52b-6f8d901ce644"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:34:40.273978 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.273892    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-kube-api-access-s9kqh" (OuterVolumeSpecName: "kube-api-access-s9kqh") pod "7f8e5763-e2fe-42b7-a52b-6f8d901ce644" (UID: "7f8e5763-e2fe-42b7-a52b-6f8d901ce644"). InnerVolumeSpecName "kube-api-access-s9kqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:34:40.373412 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.373376    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9kqh\" (UniqueName: \"kubernetes.io/projected/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-kube-api-access-s9kqh\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:40.373412 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.373407    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-home\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:40.373412 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.373417    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-tls-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:40.373663 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:40.373447    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f8e5763-e2fe-42b7-a52b-6f8d901ce644-dshm\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:34:41.086282 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:41.086233    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"
Sep 11 19:34:41.086282 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:41.086260    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds" event={"ID":"7f8e5763-e2fe-42b7-a52b-6f8d901ce644","Type":"ContainerDied","Data":"5e12ca4b9a06c26e905827010d6dbef1814794c79c5eadde2dc898f3b71028c0"}
Sep 11 19:34:41.086828 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:41.086311    2583 scope.go:117] "RemoveContainer" containerID="9fcb879d1bacaf9a0773c5c3383370390eabe1b657fbe78990b5537b7b5598ba"
Sep 11 19:34:41.101235 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:41.101206    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"]
Sep 11 19:34:41.104853 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:41.104827    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-8fbb58/simulator-w4-8fbb58-kserve-57646df67b-6jxds"]
Sep 11 19:34:42.546472 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:42.546436    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f8e5763-e2fe-42b7-a52b-6f8d901ce644" path="/var/lib/kubelet/pods/7f8e5763-e2fe-42b7-a52b-6f8d901ce644/volumes"
Sep 11 19:34:46.127745 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:46.127712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:46.895734 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:46.895702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:49.419671 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:49.419627    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:49.519939 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:49.519910    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:52.549165 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:52.549133    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:52.657547 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:52.657503    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:52.774305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:52.774220    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:52.929255 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:52.929193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:53.584432 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:53.584395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:53.683563 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:53.683519    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:54.479395 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:54.479351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:54.707435 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:54.707398    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:55.127096 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:55.127063    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:55.227772 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:55.227740    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:55.435661 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:55.435562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:55.668969 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:55.668925    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:58.562790 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:58.562745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:58.798955 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:58.798921    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:58.896458 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:58.896412    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:34:59.428652 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:34:59.428578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:00.233673 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:00.233631    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:01.150836 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.150803    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:01.207560 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.207517    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:01.462128 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.462035    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:01.640929 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.640876    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:01.736760 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.736669    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:01.781715 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.781679    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"]
Sep 11 19:35:01.782221 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782202    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78d4b411-e77d-4693-8aff-fc2d20dfd9c5" containerName="payload-processing"
Sep 11 19:35:01.782291 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782225    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="78d4b411-e77d-4693-8aff-fc2d20dfd9c5" containerName="payload-processing"
Sep 11 19:35:01.782291 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782253    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26c66035-04c3-4037-b693-88ff86d1f870" containerName="maas-api"
Sep 11 19:35:01.782291 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782259    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="26c66035-04c3-4037-b693-88ff86d1f870" containerName="maas-api"
Sep 11 19:35:01.782291 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782275    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8e5763-e2fe-42b7-a52b-6f8d901ce644" containerName="main"
Sep 11 19:35:01.782291 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782280    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8e5763-e2fe-42b7-a52b-6f8d901ce644" containerName="main"
Sep 11 19:35:01.782291 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782290    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad982c2-6fce-403a-a481-7cf8e70f334a" containerName="maas-api"
Sep 11 19:35:01.782519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782295    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad982c2-6fce-403a-a481-7cf8e70f334a" containerName="maas-api"
Sep 11 19:35:01.782519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782364    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="26c66035-04c3-4037-b693-88ff86d1f870" containerName="maas-api"
Sep 11 19:35:01.782519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782370    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ad982c2-6fce-403a-a481-7cf8e70f334a" containerName="maas-api"
Sep 11 19:35:01.782519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782376    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8e5763-e2fe-42b7-a52b-6f8d901ce644" containerName="main"
Sep 11 19:35:01.782519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.782381    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="78d4b411-e77d-4693-8aff-fc2d20dfd9c5" containerName="payload-processing"
Sep 11 19:35:01.785813 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.785796    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:35:01.787996 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.787968    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-472add-dockercfg-j8bbx\""
Sep 11 19:35:01.787996 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.787983    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-472add\""
Sep 11 19:35:01.795795 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.795760    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"]
Sep 11 19:35:01.831652 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.831619    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:01.876598 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.876568    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78365799-32d8-407e-a84d-4570ec2b7b34-plugins-config-volume\") pod \"payload-processing-e2e-explicit-472add-5545974f7-87k56\" (UID: \"78365799-32d8-407e-a84d-4570ec2b7b34\") " pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:35:01.876759 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.876604    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bqcn\" (UniqueName: \"kubernetes.io/projected/78365799-32d8-407e-a84d-4570ec2b7b34-kube-api-access-4bqcn\") pod \"payload-processing-e2e-explicit-472add-5545974f7-87k56\" (UID: \"78365799-32d8-407e-a84d-4570ec2b7b34\") " pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:35:01.977922 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.977877    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78365799-32d8-407e-a84d-4570ec2b7b34-plugins-config-volume\") pod \"payload-processing-e2e-explicit-472add-5545974f7-87k56\" (UID: \"78365799-32d8-407e-a84d-4570ec2b7b34\") " pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:35:01.977922 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.977924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqcn\" (UniqueName: \"kubernetes.io/projected/78365799-32d8-407e-a84d-4570ec2b7b34-kube-api-access-4bqcn\") pod \"payload-processing-e2e-explicit-472add-5545974f7-87k56\" (UID: \"78365799-32d8-407e-a84d-4570ec2b7b34\") " pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:35:01.978663 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.978638    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78365799-32d8-407e-a84d-4570ec2b7b34-plugins-config-volume\") pod \"payload-processing-e2e-explicit-472add-5545974f7-87k56\" (UID: \"78365799-32d8-407e-a84d-4570ec2b7b34\") " pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:35:01.986297 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:01.986275    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bqcn\" (UniqueName: \"kubernetes.io/projected/78365799-32d8-407e-a84d-4570ec2b7b34-kube-api-access-4bqcn\") pod \"payload-processing-e2e-explicit-472add-5545974f7-87k56\" (UID: \"78365799-32d8-407e-a84d-4570ec2b7b34\") " pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:35:02.098073 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:02.098035    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:35:02.373794 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:02.373767    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"]
Sep 11 19:35:02.376488 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:35:02.376461    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78365799_32d8_407e_a84d_4570ec2b7b34.slice/crio-4580415aa7079ec753cd3d8d635291d415dcab7c302f2aa8e102c1f5f08d42b8 WatchSource:0}: Error finding container 4580415aa7079ec753cd3d8d635291d415dcab7c302f2aa8e102c1f5f08d42b8: Status 404 returned error can't find the container with id 4580415aa7079ec753cd3d8d635291d415dcab7c302f2aa8e102c1f5f08d42b8
Sep 11 19:35:02.427972 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:02.427940    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:02.656003 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:02.655909    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:02.920417 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:02.920322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:03.195260 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:03.195140    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56" event={"ID":"78365799-32d8-407e-a84d-4570ec2b7b34","Type":"ContainerStarted","Data":"c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb"}
Sep 11 19:35:03.195439 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:03.195270    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:35:03.195439 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:03.195317    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56" event={"ID":"78365799-32d8-407e-a84d-4570ec2b7b34","Type":"ContainerStarted","Data":"4580415aa7079ec753cd3d8d635291d415dcab7c302f2aa8e102c1f5f08d42b8"}
Sep 11 19:35:03.211825 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:03.211779    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56" podStartSLOduration=2.211761888 podStartE2EDuration="2.211761888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:35:03.210159643 +0000 UTC m=+1007.138784541" watchObservedRunningTime="2026-09-11 19:35:03.211761888 +0000 UTC m=+1007.140386785"
Sep 11 19:35:08.431411 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:08.431363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:08.534473 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:08.534444    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:08.745140 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:08.745047    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:08.994566 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:08.994540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:09.202649 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:09.202618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:14.203206 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:14.203169    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:35:14.421694 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:14.421660    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:15.072940 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:15.072897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:15.201686 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:15.201651    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:15.334543 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:15.334460    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:16.228810 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:16.228771    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:16.529715 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:16.529615    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:17.152333 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:17.152304    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:17.943549 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:17.943523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:18.637253 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:18.637207    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:18.876421 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:18.876377    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"]
Sep 11 19:35:18.882141 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:18.882112    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:18.884462 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:18.884443    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-20a8f069-serving-cert\""
Sep 11 19:35:18.895478 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:18.895454    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"]
Sep 11 19:35:18.919595 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:18.919553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:19.026572 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.026541    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:19.056155 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.056126    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-maas-api-tls\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.056155 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.056157    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-metrics-certs\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.056387 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.056263    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef6e832-73a9-413f-8db3-2cdfdcb509de-openshift-service-ca\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.056387 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.056310    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md8fz\" (UniqueName: \"kubernetes.io/projected/eef6e832-73a9-413f-8db3-2cdfdcb509de-kube-api-access-md8fz\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.126840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.126809    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:19.156916 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.156822    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-maas-api-tls\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.156916 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.156867    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-metrics-certs\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.157151 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.156944    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef6e832-73a9-413f-8db3-2cdfdcb509de-openshift-service-ca\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.157151 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.157002    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md8fz\" (UniqueName: \"kubernetes.io/projected/eef6e832-73a9-413f-8db3-2cdfdcb509de-kube-api-access-md8fz\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.157715 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.157691    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef6e832-73a9-413f-8db3-2cdfdcb509de-openshift-service-ca\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.159809 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.159786    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-metrics-certs\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.159907 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.159789    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-maas-api-tls\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.165309 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.165284    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md8fz\" (UniqueName: \"kubernetes.io/projected/eef6e832-73a9-413f-8db3-2cdfdcb509de-kube-api-access-md8fz\") pod \"maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.194719 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.194688    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:19.476148 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.476115    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"]
Sep 11 19:35:19.478085 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:35:19.478036    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef6e832_73a9_413f_8db3_2cdfdcb509de.slice/crio-dc1222619519f5d03f6ff47847acd75b19ad7a7a53f3859f1a4e70f6eba356c9 WatchSource:0}: Error finding container dc1222619519f5d03f6ff47847acd75b19ad7a7a53f3859f1a4e70f6eba356c9: Status 404 returned error can't find the container with id dc1222619519f5d03f6ff47847acd75b19ad7a7a53f3859f1a4e70f6eba356c9
Sep 11 19:35:19.479829 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.479806    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 19:35:19.512420 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.512383    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:19.617644 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.617606    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:19.628017 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.627976    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"]
Sep 11 19:35:19.632946 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.632929    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:35:19.635697 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.635677    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-20a8f069-dockercfg-8lhjk\""
Sep 11 19:35:19.636752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.636726    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-20a8f069\""
Sep 11 19:35:19.644680 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.644660    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"]
Sep 11 19:35:19.763777 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.763692    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg8jk\" (UniqueName: \"kubernetes.io/projected/902958c4-aaef-4357-badc-87ccd6198039-kube-api-access-zg8jk\") pod \"payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd\" (UID: \"902958c4-aaef-4357-badc-87ccd6198039\") " pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:35:19.763777 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.763739    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/902958c4-aaef-4357-badc-87ccd6198039-plugins-config-volume\") pod \"payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd\" (UID: \"902958c4-aaef-4357-badc-87ccd6198039\") " pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:35:19.864851 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.864814    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg8jk\" (UniqueName: \"kubernetes.io/projected/902958c4-aaef-4357-badc-87ccd6198039-kube-api-access-zg8jk\") pod \"payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd\" (UID: \"902958c4-aaef-4357-badc-87ccd6198039\") " pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:35:19.864980 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.864880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/902958c4-aaef-4357-badc-87ccd6198039-plugins-config-volume\") pod \"payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd\" (UID: \"902958c4-aaef-4357-badc-87ccd6198039\") " pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:35:19.865581 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.865558    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/902958c4-aaef-4357-badc-87ccd6198039-plugins-config-volume\") pod \"payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd\" (UID: \"902958c4-aaef-4357-badc-87ccd6198039\") " pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:35:19.874661 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.874641    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg8jk\" (UniqueName: \"kubernetes.io/projected/902958c4-aaef-4357-badc-87ccd6198039-kube-api-access-zg8jk\") pod \"payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd\" (UID: \"902958c4-aaef-4357-badc-87ccd6198039\") " pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:35:19.945064 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:19.945031    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:35:20.046266 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:20.046153    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:20.228591 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:20.228566    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"]
Sep 11 19:35:20.230721 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:35:20.230690    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902958c4_aaef_4357_badc_87ccd6198039.slice/crio-73a4674ff1fe2bc72eb790f7c7d5ed9d40a876ceacc86efa57dc0e99b4613d02 WatchSource:0}: Error finding container 73a4674ff1fe2bc72eb790f7c7d5ed9d40a876ceacc86efa57dc0e99b4613d02: Status 404 returned error can't find the container with id 73a4674ff1fe2bc72eb790f7c7d5ed9d40a876ceacc86efa57dc0e99b4613d02
Sep 11 19:35:20.292289 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:20.292260    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd" event={"ID":"902958c4-aaef-4357-badc-87ccd6198039","Type":"ContainerStarted","Data":"73a4674ff1fe2bc72eb790f7c7d5ed9d40a876ceacc86efa57dc0e99b4613d02"}
Sep 11 19:35:20.293825 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:20.293798    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57" event={"ID":"eef6e832-73a9-413f-8db3-2cdfdcb509de","Type":"ContainerStarted","Data":"66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27"}
Sep 11 19:35:20.293910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:20.293850    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57" event={"ID":"eef6e832-73a9-413f-8db3-2cdfdcb509de","Type":"ContainerStarted","Data":"dc1222619519f5d03f6ff47847acd75b19ad7a7a53f3859f1a4e70f6eba356c9"}
Sep 11 19:35:20.293910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:20.293890    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:20.313305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:20.313164    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57" podStartSLOduration=1.949543665 podStartE2EDuration="2.313147997s" totalImagesPullingTime="363.604332ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:35:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:35:20.311107475 +0000 UTC m=+1024.239732373" watchObservedRunningTime="2026-09-11 19:35:20.313147997 +0000 UTC m=+1024.241772896"
Sep 11 19:35:20.873351 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:20.873310    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:21.300381 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:21.300279    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd" event={"ID":"902958c4-aaef-4357-badc-87ccd6198039","Type":"ContainerStarted","Data":"bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83"}
Sep 11 19:35:21.300381 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:21.300359    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:35:21.316940 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:21.316889    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd" podStartSLOduration=2.316872318 podStartE2EDuration="2.316872318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:35:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:35:21.31499352 +0000 UTC m=+1025.243618418" watchObservedRunningTime="2026-09-11 19:35:21.316872318 +0000 UTC m=+1025.245497217"
Sep 11 19:35:21.687904 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:21.687864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:22.567553 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:22.567524    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:23.260541 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:23.260511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:24.042809 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:24.042778    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:24.869298 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:24.869266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:26.306051 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:26.306023    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:35:32.307965 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:32.307923    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:35:36.335666 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:36.335637    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:40.017022 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:40.016981    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:41.239711 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:41.239651    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:42.004507 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:42.004471    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:42.753591 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:42.753527    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:43.437767 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:43.437729    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:44.093392 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:44.093362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:44.951642 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:44.951607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:45.095374 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:45.095338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:45.754768 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:45.754730    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:45.852231 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:45.852199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:46.227452 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:46.227421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:51.930700 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:51.930669    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:51.993443 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:51.993405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:53.043772 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:53.043735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:53.606665 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:53.606625    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:53.707765 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:53.707736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:53.810446 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:53.810416    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:54.565258 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:54.565212    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:55.271813 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:55.271781    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:56.244831 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:56.244800    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:35:56.960127 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:35:56.960094    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:02.867990 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:02.867944    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"]
Sep 11 19:36:02.868482 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:02.868270    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57" podUID="eef6e832-73a9-413f-8db3-2cdfdcb509de" containerName="maas-api" containerID="cri-o://66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27" gracePeriod=30
Sep 11 19:36:02.897556 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:02.897519    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:02.921643 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:02.921601    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"]
Sep 11 19:36:02.921947 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:02.921886    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd" podUID="902958c4-aaef-4357-badc-87ccd6198039" containerName="payload-processing" containerID="cri-o://bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83" gracePeriod=30
Sep 11 19:36:02.961988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:02.961959    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:03.063297 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.063232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:03.117958 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.117926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:03.181325 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.181297    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:36:03.216908 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.216882    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:36:03.288016 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.287986    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-maas-api-tls\") pod \"eef6e832-73a9-413f-8db3-2cdfdcb509de\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") "
Sep 11 19:36:03.288016 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.288021    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-metrics-certs\") pod \"eef6e832-73a9-413f-8db3-2cdfdcb509de\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") "
Sep 11 19:36:03.288319 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.288042    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef6e832-73a9-413f-8db3-2cdfdcb509de-openshift-service-ca\") pod \"eef6e832-73a9-413f-8db3-2cdfdcb509de\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") "
Sep 11 19:36:03.288319 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.288098    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-md8fz\" (UniqueName: \"kubernetes.io/projected/eef6e832-73a9-413f-8db3-2cdfdcb509de-kube-api-access-md8fz\") pod \"eef6e832-73a9-413f-8db3-2cdfdcb509de\" (UID: \"eef6e832-73a9-413f-8db3-2cdfdcb509de\") "
Sep 11 19:36:03.288542 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.288511    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eef6e832-73a9-413f-8db3-2cdfdcb509de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eef6e832-73a9-413f-8db3-2cdfdcb509de" (UID: "eef6e832-73a9-413f-8db3-2cdfdcb509de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:36:03.290378 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.290348    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eef6e832-73a9-413f-8db3-2cdfdcb509de" (UID: "eef6e832-73a9-413f-8db3-2cdfdcb509de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:36:03.290378 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.290361    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eef6e832-73a9-413f-8db3-2cdfdcb509de" (UID: "eef6e832-73a9-413f-8db3-2cdfdcb509de"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:36:03.290497 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.290375    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eef6e832-73a9-413f-8db3-2cdfdcb509de-kube-api-access-md8fz" (OuterVolumeSpecName: "kube-api-access-md8fz") pod "eef6e832-73a9-413f-8db3-2cdfdcb509de" (UID: "eef6e832-73a9-413f-8db3-2cdfdcb509de"). InnerVolumeSpecName "kube-api-access-md8fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:36:03.389660 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.389566    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg8jk\" (UniqueName: \"kubernetes.io/projected/902958c4-aaef-4357-badc-87ccd6198039-kube-api-access-zg8jk\") pod \"902958c4-aaef-4357-badc-87ccd6198039\" (UID: \"902958c4-aaef-4357-badc-87ccd6198039\") "
Sep 11 19:36:03.389830 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.389711    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/902958c4-aaef-4357-badc-87ccd6198039-plugins-config-volume\") pod \"902958c4-aaef-4357-badc-87ccd6198039\" (UID: \"902958c4-aaef-4357-badc-87ccd6198039\") "
Sep 11 19:36:03.389897 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.389890    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:36:03.389954 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.389903    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef6e832-73a9-413f-8db3-2cdfdcb509de-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:36:03.389954 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.389913    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef6e832-73a9-413f-8db3-2cdfdcb509de-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:36:03.389954 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.389921    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md8fz\" (UniqueName: \"kubernetes.io/projected/eef6e832-73a9-413f-8db3-2cdfdcb509de-kube-api-access-md8fz\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:36:03.390078 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.390014    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/902958c4-aaef-4357-badc-87ccd6198039-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "902958c4-aaef-4357-badc-87ccd6198039" (UID: "902958c4-aaef-4357-badc-87ccd6198039"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:36:03.392012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.391979    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/902958c4-aaef-4357-badc-87ccd6198039-kube-api-access-zg8jk" (OuterVolumeSpecName: "kube-api-access-zg8jk") pod "902958c4-aaef-4357-badc-87ccd6198039" (UID: "902958c4-aaef-4357-badc-87ccd6198039"). InnerVolumeSpecName "kube-api-access-zg8jk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:36:03.399924 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.399895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:03.491045 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.491011    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/902958c4-aaef-4357-badc-87ccd6198039-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:36:03.491045 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.491037    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg8jk\" (UniqueName: \"kubernetes.io/projected/902958c4-aaef-4357-badc-87ccd6198039-kube-api-access-zg8jk\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:36:03.512367 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.512336    2583 generic.go:356] "Generic (PLEG): container finished" podID="eef6e832-73a9-413f-8db3-2cdfdcb509de" containerID="66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27" exitCode=0
Sep 11 19:36:03.512537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.512375    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"
Sep 11 19:36:03.512537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.512425    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57" event={"ID":"eef6e832-73a9-413f-8db3-2cdfdcb509de","Type":"ContainerDied","Data":"66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27"}
Sep 11 19:36:03.512537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.512485    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57" event={"ID":"eef6e832-73a9-413f-8db3-2cdfdcb509de","Type":"ContainerDied","Data":"dc1222619519f5d03f6ff47847acd75b19ad7a7a53f3859f1a4e70f6eba356c9"}
Sep 11 19:36:03.512537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.512525    2583 scope.go:117] "RemoveContainer" containerID="66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27"
Sep 11 19:36:03.514060 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.514022    2583 generic.go:356] "Generic (PLEG): container finished" podID="902958c4-aaef-4357-badc-87ccd6198039" containerID="bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83" exitCode=0
Sep 11 19:36:03.514180 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.514057    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd" event={"ID":"902958c4-aaef-4357-badc-87ccd6198039","Type":"ContainerDied","Data":"bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83"}
Sep 11 19:36:03.514180 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.514074    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"
Sep 11 19:36:03.514319 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.514226    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd" event={"ID":"902958c4-aaef-4357-badc-87ccd6198039","Type":"ContainerDied","Data":"73a4674ff1fe2bc72eb790f7c7d5ed9d40a876ceacc86efa57dc0e99b4613d02"}
Sep 11 19:36:03.524681 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.524661    2583 scope.go:117] "RemoveContainer" containerID="66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27"
Sep 11 19:36:03.524968 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:36:03.524947    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27\": container with ID starting with 66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27 not found: ID does not exist" containerID="66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27"
Sep 11 19:36:03.525028 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.524979    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27"} err="failed to get container status \"66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27\": rpc error: code = NotFound desc = could not find container \"66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27\": container with ID starting with 66c8221242073d2dd1e797bb2ce622b934f442c6de17a4c893d23cbf10a7ed27 not found: ID does not exist"
Sep 11 19:36:03.525028 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.525001    2583 scope.go:117] "RemoveContainer" containerID="bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83"
Sep 11 19:36:03.533355 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.533332    2583 scope.go:117] "RemoveContainer" containerID="bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83"
Sep 11 19:36:03.533635 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:36:03.533618    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83\": container with ID starting with bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83 not found: ID does not exist" containerID="bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83"
Sep 11 19:36:03.533693 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.533639    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83"} err="failed to get container status \"bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83\": rpc error: code = NotFound desc = could not find container \"bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83\": container with ID starting with bc4098e9e29dd8512a9a98922c14f930b42ebeda4ae6982e1f98c78b21ae2d83 not found: ID does not exist"
Sep 11 19:36:03.539760 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.539715    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"]
Sep 11 19:36:03.542133 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.542109    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20a8f069-6b49d68c97-mlk57"]
Sep 11 19:36:03.542225 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.542210    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:03.551790 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.551768    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"]
Sep 11 19:36:03.555650 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:03.555632    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20a8f069-6b99f87bfc-6kmfd"]
Sep 11 19:36:04.257192 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:04.257110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:04.547394 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:04.547232    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="902958c4-aaef-4357-badc-87ccd6198039" path="/var/lib/kubelet/pods/902958c4-aaef-4357-badc-87ccd6198039/volumes"
Sep 11 19:36:04.547849 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:04.547811    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eef6e832-73a9-413f-8db3-2cdfdcb509de" path="/var/lib/kubelet/pods/eef6e832-73a9-413f-8db3-2cdfdcb509de/volumes"
Sep 11 19:36:05.033844 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:05.033816    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:05.738094 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:05.738064    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:06.522722 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:06.522688    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:09.311118 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:09.311027    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:09.418718 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:09.418683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:09.568867 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:09.568790    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:09.925713 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:09.925486    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:10.019802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:10.019768    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:10.129163 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:10.129131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:10.575555 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:10.575521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:15.838779 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:15.838749    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:15.904941 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:15.904910    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:15.985396 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:15.985364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:16.044137 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:16.044098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:16.179500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:16.179463    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:18.812877 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.812835    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"]
Sep 11 19:36:18.813456 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.813437    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="902958c4-aaef-4357-badc-87ccd6198039" containerName="payload-processing"
Sep 11 19:36:18.813541 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.813460    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="902958c4-aaef-4357-badc-87ccd6198039" containerName="payload-processing"
Sep 11 19:36:18.813541 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.813474    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eef6e832-73a9-413f-8db3-2cdfdcb509de" containerName="maas-api"
Sep 11 19:36:18.813541 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.813482    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="eef6e832-73a9-413f-8db3-2cdfdcb509de" containerName="maas-api"
Sep 11 19:36:18.813698 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.813597    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="902958c4-aaef-4357-badc-87ccd6198039" containerName="payload-processing"
Sep 11 19:36:18.813698 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.813621    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="eef6e832-73a9-413f-8db3-2cdfdcb509de" containerName="maas-api"
Sep 11 19:36:18.818174 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.818152    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:18.821284 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.821259    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-472add\"/\"explicit-model-472add-kserve-self-signed-certs\""
Sep 11 19:36:18.821431 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.821285    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-472add\"/\"kube-root-ca.crt\""
Sep 11 19:36:18.821431 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.821266    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-472add\"/\"default-dockercfg-z8p4q\""
Sep 11 19:36:18.821431 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.821367    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-472add\"/\"openshift-service-ca.crt\""
Sep 11 19:36:18.824441 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.824419    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"]
Sep 11 19:36:18.841982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.841951    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:18.927693 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.927659    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-model-cache\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:18.927891 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.927728    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40525483-a300-487a-8f01-633b4e04be7a-tls-certs\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:18.927891 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.927838    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl8xm\" (UniqueName: \"kubernetes.io/projected/40525483-a300-487a-8f01-633b4e04be7a-kube-api-access-cl8xm\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:18.927891 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.927878    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-dshm\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:18.928045 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.927944    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-tmp-dir\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:18.928045 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:18.927974    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-home\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.029511 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.029470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-model-cache\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.029687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.029558    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40525483-a300-487a-8f01-633b4e04be7a-tls-certs\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.029687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.029620    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl8xm\" (UniqueName: \"kubernetes.io/projected/40525483-a300-487a-8f01-633b4e04be7a-kube-api-access-cl8xm\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.029687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.029652    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-dshm\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.029687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.029681    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-tmp-dir\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.029846 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.029815    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-home\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.029933 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.029912    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-model-cache\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.030088 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.030070    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-tmp-dir\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.030165 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.030112    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-home\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.031893 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.031873    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-dshm\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.032154 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.032134    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40525483-a300-487a-8f01-633b4e04be7a-tls-certs\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.037302 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.037285    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl8xm\" (UniqueName: \"kubernetes.io/projected/40525483-a300-487a-8f01-633b4e04be7a-kube-api-access-cl8xm\") pod \"explicit-model-472add-kserve-66c997ffd9-dx2rr\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") " pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.129227 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.129183    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:19.237373 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.237340    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:19.402370 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.402157    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"]
Sep 11 19:36:19.405212 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:36:19.405177    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40525483_a300_487a_8f01_633b4e04be7a.slice/crio-4343653bb9e61c2d0669eb9e1a321be72a4553557d25aaa39ae8d398f233d915 WatchSource:0}: Error finding container 4343653bb9e61c2d0669eb9e1a321be72a4553557d25aaa39ae8d398f233d915: Status 404 returned error can't find the container with id 4343653bb9e61c2d0669eb9e1a321be72a4553557d25aaa39ae8d398f233d915
Sep 11 19:36:19.595262 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.595216    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr" event={"ID":"40525483-a300-487a-8f01-633b4e04be7a","Type":"ContainerStarted","Data":"08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda"}
Sep 11 19:36:19.595442 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.595293    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr" event={"ID":"40525483-a300-487a-8f01-633b4e04be7a","Type":"ContainerStarted","Data":"4343653bb9e61c2d0669eb9e1a321be72a4553557d25aaa39ae8d398f233d915"}
Sep 11 19:36:19.613548 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:19.613493    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr" podStartSLOduration=1.6134757880000001 podStartE2EDuration="1.613475788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:36:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:36:19.611747798 +0000 UTC m=+1083.540372697" watchObservedRunningTime="2026-09-11 19:36:19.613475788 +0000 UTC m=+1083.542100685"
Sep 11 19:36:21.038959 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:21.038924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:22.047585 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:22.047548    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:22.147227 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:22.147188    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:22.939935 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:22.939900    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:23.221658 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:23.221558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:23.350520 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:23.350482    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:25.913547 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:25.913510    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:26.616228 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:26.616201    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:26.805868 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:26.805838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:27.738189 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:27.738157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:28.551877 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:28.551831    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:29.129966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:29.129925    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:29.130398 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:29.129984    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:29.142716 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:29.142690    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:29.250579 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:29.250536    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:29.658555 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:29.658528    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:36:29.909394 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:29.909316    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"]
Sep 11 19:36:29.913084 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:29.913062    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:29.916087 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:29.916065    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-b5bf35-serving-cert\""
Sep 11 19:36:29.927443 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:29.927419    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"]
Sep 11 19:36:29.949672 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:29.949635    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:30.031016 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.030977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.031214 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.031042    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.031214 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.031085    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8dgv\" (UniqueName: \"kubernetes.io/projected/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-kube-api-access-q8dgv\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.031214 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.031202    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.047077 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.047035    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:30.131763 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.131727    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.132190 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.131780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.132190 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.131823    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.132190 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.131849    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8dgv\" (UniqueName: \"kubernetes.io/projected/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-kube-api-access-q8dgv\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.132190 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:36:30.131962    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-serving-cert: secret "maas-api-e2e-shared-a-w2-b5bf35-serving-cert" not found
Sep 11 19:36:30.132190 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:36:30.132028    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-maas-api-tls podName:5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3 nodeName:}" failed. No retries permitted until 2026-09-11 19:36:30.632008734 +0000 UTC m=+1094.560633611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-maas-api-tls") pod "maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6" (UID: "5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3") : secret "maas-api-e2e-shared-a-w2-b5bf35-serving-cert" not found
Sep 11 19:36:30.132522 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.132503    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.134470 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.134434    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.140415 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.140393    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8dgv\" (UniqueName: \"kubernetes.io/projected/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-kube-api-access-q8dgv\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.222341 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.222218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:30.460402 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.460369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:30.557565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.557489    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:30.636488 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.636452    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.639155 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.639135    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:30.826034 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:30.825943    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:31.028417 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:31.028370    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:31.102804 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:31.102772    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"]
Sep 11 19:36:31.104733 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:36:31.104704    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e4d73a5_29c3_4bbe_a0ae_e4e381b4a4c3.slice/crio-0e47941b9897c2c85ae1ca58306b849aca383865e6e041ff621d7110b14b2d79 WatchSource:0}: Error finding container 0e47941b9897c2c85ae1ca58306b849aca383865e6e041ff621d7110b14b2d79: Status 404 returned error can't find the container with id 0e47941b9897c2c85ae1ca58306b849aca383865e6e041ff621d7110b14b2d79
Sep 11 19:36:31.659587 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:31.659543    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6" event={"ID":"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3","Type":"ContainerStarted","Data":"f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb"}
Sep 11 19:36:31.660081 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:31.659648    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:31.660081 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:31.659670    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6" event={"ID":"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3","Type":"ContainerStarted","Data":"0e47941b9897c2c85ae1ca58306b849aca383865e6e041ff621d7110b14b2d79"}
Sep 11 19:36:31.677148 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:31.677090    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6" podStartSLOduration=2.326242368 podStartE2EDuration="2.677070324s" totalImagesPullingTime="350.827956ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:36:31.676963264 +0000 UTC m=+1095.605588163" watchObservedRunningTime="2026-09-11 19:36:31.677070324 +0000 UTC m=+1095.605695222"
Sep 11 19:36:37.670416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:37.670385    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:36:40.639640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:40.639607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:51.900683 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:51.900645    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:52.718791 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:52.718751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:53.374136 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:53.374103    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:54.039599 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:54.039553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:54.692695 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:54.692660    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:54.752156 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:54.752124    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:54.843162 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:54.843121    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:55.535229 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:55.535195    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:56.310701 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:56.310670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:56.625829 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:56.625785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:57.025795 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:57.025667    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:57.148447 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:57.148393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:57.323934 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:57.323862    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:36:57.617591 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:36:57.617561    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:00.226286 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:00.226236    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:00.796634 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:00.796599    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:00.898786 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:00.898751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:01.300010 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:01.299972    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:01.991208 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:01.991174    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:02.706488 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:02.706423    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:02.773398 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:02.773363    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"]
Sep 11 19:37:02.773599 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:02.773569    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56" podUID="78365799-32d8-407e-a84d-4570ec2b7b34" containerName="payload-processing" containerID="cri-o://c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb" gracePeriod=30
Sep 11 19:37:02.838941 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:02.838904    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:02.939201 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:02.939169    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:03.036486 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.036461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:03.044679 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.044658    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:37:03.151139 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.151107    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bqcn\" (UniqueName: \"kubernetes.io/projected/78365799-32d8-407e-a84d-4570ec2b7b34-kube-api-access-4bqcn\") pod \"78365799-32d8-407e-a84d-4570ec2b7b34\" (UID: \"78365799-32d8-407e-a84d-4570ec2b7b34\") "
Sep 11 19:37:03.151365 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.151195    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78365799-32d8-407e-a84d-4570ec2b7b34-plugins-config-volume\") pod \"78365799-32d8-407e-a84d-4570ec2b7b34\" (UID: \"78365799-32d8-407e-a84d-4570ec2b7b34\") "
Sep 11 19:37:03.151623 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.151593    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78365799-32d8-407e-a84d-4570ec2b7b34-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "78365799-32d8-407e-a84d-4570ec2b7b34" (UID: "78365799-32d8-407e-a84d-4570ec2b7b34"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:37:03.153649 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.153612    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78365799-32d8-407e-a84d-4570ec2b7b34-kube-api-access-4bqcn" (OuterVolumeSpecName: "kube-api-access-4bqcn") pod "78365799-32d8-407e-a84d-4570ec2b7b34" (UID: "78365799-32d8-407e-a84d-4570ec2b7b34"). InnerVolumeSpecName "kube-api-access-4bqcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:37:03.209360 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.209329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:03.252477 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.252442    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78365799-32d8-407e-a84d-4570ec2b7b34-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:03.252477 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.252470    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bqcn\" (UniqueName: \"kubernetes.io/projected/78365799-32d8-407e-a84d-4570ec2b7b34-kube-api-access-4bqcn\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:03.339977 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.339885    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:03.830104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.830067    2583 generic.go:356] "Generic (PLEG): container finished" podID="78365799-32d8-407e-a84d-4570ec2b7b34" containerID="c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb" exitCode=0
Sep 11 19:37:03.830569 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.830110    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56" event={"ID":"78365799-32d8-407e-a84d-4570ec2b7b34","Type":"ContainerDied","Data":"c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb"}
Sep 11 19:37:03.830569 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.830151    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56" event={"ID":"78365799-32d8-407e-a84d-4570ec2b7b34","Type":"ContainerDied","Data":"4580415aa7079ec753cd3d8d635291d415dcab7c302f2aa8e102c1f5f08d42b8"}
Sep 11 19:37:03.830569 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.830172    2583 scope.go:117] "RemoveContainer" containerID="c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb"
Sep 11 19:37:03.830569 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.830113    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"
Sep 11 19:37:03.839423 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.839404    2583 scope.go:117] "RemoveContainer" containerID="c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb"
Sep 11 19:37:03.839716 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:03.839698    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb\": container with ID starting with c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb not found: ID does not exist" containerID="c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb"
Sep 11 19:37:03.839777 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.839725    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb"} err="failed to get container status \"c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb\": rpc error: code = NotFound desc = could not find container \"c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb\": container with ID starting with c300c816ec59659dc54e2ee6366810d5c93acbda5d4f88f00725fa529ecac0bb not found: ID does not exist"
Sep 11 19:37:03.853635 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.853601    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"]
Sep 11 19:37:03.855526 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.855502    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-472add-5545974f7-87k56"]
Sep 11 19:37:03.937953 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:03.937913    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:04.546342 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:04.546266    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78365799-32d8-407e-a84d-4570ec2b7b34" path="/var/lib/kubelet/pods/78365799-32d8-407e-a84d-4570ec2b7b34/volumes"
Sep 11 19:37:04.897068 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:04.897029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:04.998389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:04.998358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:05.058559 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.058523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:05.506290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.506232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:05.606075 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.606040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:05.615149 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.615126    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"]
Sep 11 19:37:05.615570 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.615557    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78365799-32d8-407e-a84d-4570ec2b7b34" containerName="payload-processing"
Sep 11 19:37:05.615624 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.615572    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="78365799-32d8-407e-a84d-4570ec2b7b34" containerName="payload-processing"
Sep 11 19:37:05.615660 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.615648    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="78365799-32d8-407e-a84d-4570ec2b7b34" containerName="payload-processing"
Sep 11 19:37:05.620088 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.620068    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:37:05.622586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.622563    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-b20815-dockercfg-jwpc2\""
Sep 11 19:37:05.622735 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.622707    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-b20815\""
Sep 11 19:37:05.631259 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.631221    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"]
Sep 11 19:37:05.645660 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.645637    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"]
Sep 11 19:37:05.649578 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.649556    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:37:05.660664 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.660641    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"]
Sep 11 19:37:05.675521 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.675483    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dd0b9d1-a040-4b61-9bac-30413990044e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9\" (UID: \"8dd0b9d1-a040-4b61-9bac-30413990044e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:37:05.675679 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.675581    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mks55\" (UniqueName: \"kubernetes.io/projected/8dd0b9d1-a040-4b61-9bac-30413990044e-kube-api-access-mks55\") pod \"payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9\" (UID: \"8dd0b9d1-a040-4b61-9bac-30413990044e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:37:05.777151 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.777075    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dd0b9d1-a040-4b61-9bac-30413990044e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9\" (UID: \"8dd0b9d1-a040-4b61-9bac-30413990044e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:37:05.777151 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.777131    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01b47d19-c133-4339-820e-56a8521a4994-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7\" (UID: \"01b47d19-c133-4339-820e-56a8521a4994\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:37:05.777354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.777165    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mks55\" (UniqueName: \"kubernetes.io/projected/8dd0b9d1-a040-4b61-9bac-30413990044e-kube-api-access-mks55\") pod \"payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9\" (UID: \"8dd0b9d1-a040-4b61-9bac-30413990044e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:37:05.777354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.777190    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp9dm\" (UniqueName: \"kubernetes.io/projected/01b47d19-c133-4339-820e-56a8521a4994-kube-api-access-fp9dm\") pod \"payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7\" (UID: \"01b47d19-c133-4339-820e-56a8521a4994\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:37:05.777810 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.777787    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dd0b9d1-a040-4b61-9bac-30413990044e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9\" (UID: \"8dd0b9d1-a040-4b61-9bac-30413990044e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:37:05.787087 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.787058    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mks55\" (UniqueName: \"kubernetes.io/projected/8dd0b9d1-a040-4b61-9bac-30413990044e-kube-api-access-mks55\") pod \"payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9\" (UID: \"8dd0b9d1-a040-4b61-9bac-30413990044e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:37:05.878029 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.877991    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01b47d19-c133-4339-820e-56a8521a4994-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7\" (UID: \"01b47d19-c133-4339-820e-56a8521a4994\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:37:05.878203 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.878046    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp9dm\" (UniqueName: \"kubernetes.io/projected/01b47d19-c133-4339-820e-56a8521a4994-kube-api-access-fp9dm\") pod \"payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7\" (UID: \"01b47d19-c133-4339-820e-56a8521a4994\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:37:05.878786 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.878766    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01b47d19-c133-4339-820e-56a8521a4994-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7\" (UID: \"01b47d19-c133-4339-820e-56a8521a4994\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:37:05.886477 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.886447    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp9dm\" (UniqueName: \"kubernetes.io/projected/01b47d19-c133-4339-820e-56a8521a4994-kube-api-access-fp9dm\") pod \"payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7\" (UID: \"01b47d19-c133-4339-820e-56a8521a4994\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:37:05.931719 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.931685    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:37:05.948421 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.948387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:05.960319 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:05.960290    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:37:06.214490 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:06.214464    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"]
Sep 11 19:37:06.216667 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:37:06.216640    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd0b9d1_a040_4b61_9bac_30413990044e.slice/crio-71d324bc4d5e877c7b33ecbef8359fd54fb2cbab14e066e3718231161f4501ca WatchSource:0}: Error finding container 71d324bc4d5e877c7b33ecbef8359fd54fb2cbab14e066e3718231161f4501ca: Status 404 returned error can't find the container with id 71d324bc4d5e877c7b33ecbef8359fd54fb2cbab14e066e3718231161f4501ca
Sep 11 19:37:06.246873 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:06.246819    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"]
Sep 11 19:37:06.258494 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:37:06.258466    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b47d19_c133_4339_820e_56a8521a4994.slice/crio-6fab17796a71f8f2ff6010455341470d105826f42c29a2e6285350e4f7b8681f WatchSource:0}: Error finding container 6fab17796a71f8f2ff6010455341470d105826f42c29a2e6285350e4f7b8681f: Status 404 returned error can't find the container with id 6fab17796a71f8f2ff6010455341470d105826f42c29a2e6285350e4f7b8681f
Sep 11 19:37:06.782982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:06.782942    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:06.848680 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:06.848642    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7" event={"ID":"01b47d19-c133-4339-820e-56a8521a4994","Type":"ContainerStarted","Data":"77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54"}
Sep 11 19:37:06.848884 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:06.848867    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:37:06.848949 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:06.848899    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7" event={"ID":"01b47d19-c133-4339-820e-56a8521a4994","Type":"ContainerStarted","Data":"6fab17796a71f8f2ff6010455341470d105826f42c29a2e6285350e4f7b8681f"}
Sep 11 19:37:06.850306 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:06.850277    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9" event={"ID":"8dd0b9d1-a040-4b61-9bac-30413990044e","Type":"ContainerStarted","Data":"477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f"}
Sep 11 19:37:06.850455 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:06.850325    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9" event={"ID":"8dd0b9d1-a040-4b61-9bac-30413990044e","Type":"ContainerStarted","Data":"71d324bc4d5e877c7b33ecbef8359fd54fb2cbab14e066e3718231161f4501ca"}
Sep 11 19:37:06.850455 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:06.850390    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:37:06.870090 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:06.870045    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7" podStartSLOduration=1.870030439 podStartE2EDuration="1.870030439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:37:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:37:06.868159018 +0000 UTC m=+1130.796783915" watchObservedRunningTime="2026-09-11 19:37:06.870030439 +0000 UTC m=+1130.798655337"
Sep 11 19:37:06.884629 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:06.884526    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9" podStartSLOduration=1.88450657 podStartE2EDuration="1.88450657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:37:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:37:06.883860977 +0000 UTC m=+1130.812485877" watchObservedRunningTime="2026-09-11 19:37:06.88450657 +0000 UTC m=+1130.813131471"
Sep 11 19:37:07.543891 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:07.543844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:08.193327 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:08.193296    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:08.321526 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:08.321489    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"]
Sep 11 19:37:08.416816 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:08.416784    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:08.949704 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:08.949663    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:09.998746 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:09.998714    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:17.857650 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:17.857608    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:37:17.858057 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:17.857698    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:37:23.341945 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.341884    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr" podUID="40525483-a300-487a-8f01-633b4e04be7a" containerName="main" containerID="cri-o://08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda" gracePeriod=45
Sep 11 19:37:23.607739 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.607677    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:37:23.756151 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756123    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-home\") pod \"40525483-a300-487a-8f01-633b4e04be7a\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") "
Sep 11 19:37:23.756329 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756192    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl8xm\" (UniqueName: \"kubernetes.io/projected/40525483-a300-487a-8f01-633b4e04be7a-kube-api-access-cl8xm\") pod \"40525483-a300-487a-8f01-633b4e04be7a\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") "
Sep 11 19:37:23.756329 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756211    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-tmp-dir\") pod \"40525483-a300-487a-8f01-633b4e04be7a\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") "
Sep 11 19:37:23.756329 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756276    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-dshm\") pod \"40525483-a300-487a-8f01-633b4e04be7a\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") "
Sep 11 19:37:23.756504 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756330    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-model-cache\") pod \"40525483-a300-487a-8f01-633b4e04be7a\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") "
Sep 11 19:37:23.756504 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756360    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40525483-a300-487a-8f01-633b4e04be7a-tls-certs\") pod \"40525483-a300-487a-8f01-633b4e04be7a\" (UID: \"40525483-a300-487a-8f01-633b4e04be7a\") "
Sep 11 19:37:23.756504 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756403    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-home" (OuterVolumeSpecName: "home") pod "40525483-a300-487a-8f01-633b4e04be7a" (UID: "40525483-a300-487a-8f01-633b4e04be7a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:37:23.756657 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756563    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-model-cache" (OuterVolumeSpecName: "model-cache") pod "40525483-a300-487a-8f01-633b4e04be7a" (UID: "40525483-a300-487a-8f01-633b4e04be7a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:37:23.756657 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756580    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "40525483-a300-487a-8f01-633b4e04be7a" (UID: "40525483-a300-487a-8f01-633b4e04be7a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:37:23.756777 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756759    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-home\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:23.756874 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756783    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-tmp-dir\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:23.756874 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.756798    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-model-cache\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:23.758556 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.758528    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40525483-a300-487a-8f01-633b4e04be7a-kube-api-access-cl8xm" (OuterVolumeSpecName: "kube-api-access-cl8xm") pod "40525483-a300-487a-8f01-633b4e04be7a" (UID: "40525483-a300-487a-8f01-633b4e04be7a"). InnerVolumeSpecName "kube-api-access-cl8xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:37:23.758556 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.758548    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-dshm" (OuterVolumeSpecName: "dshm") pod "40525483-a300-487a-8f01-633b4e04be7a" (UID: "40525483-a300-487a-8f01-633b4e04be7a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:37:23.758701 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.758602    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40525483-a300-487a-8f01-633b4e04be7a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "40525483-a300-487a-8f01-633b4e04be7a" (UID: "40525483-a300-487a-8f01-633b4e04be7a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:37:23.857800 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.857718    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl8xm\" (UniqueName: \"kubernetes.io/projected/40525483-a300-487a-8f01-633b4e04be7a-kube-api-access-cl8xm\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:23.857800 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.857746    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40525483-a300-487a-8f01-633b4e04be7a-dshm\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:23.857800 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.857757    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40525483-a300-487a-8f01-633b4e04be7a-tls-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:23.939218 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.939175    2583 generic.go:356] "Generic (PLEG): container finished" podID="40525483-a300-487a-8f01-633b4e04be7a" containerID="08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda" exitCode=0
Sep 11 19:37:23.939218 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.939216    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"
Sep 11 19:37:23.939463 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.939234    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr" event={"ID":"40525483-a300-487a-8f01-633b4e04be7a","Type":"ContainerDied","Data":"08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda"}
Sep 11 19:37:23.939463 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.939316    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr" event={"ID":"40525483-a300-487a-8f01-633b4e04be7a","Type":"ContainerDied","Data":"4343653bb9e61c2d0669eb9e1a321be72a4553557d25aaa39ae8d398f233d915"}
Sep 11 19:37:23.939463 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.939333    2583 scope.go:117] "RemoveContainer" containerID="08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda"
Sep 11 19:37:23.949120 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.949102    2583 scope.go:117] "RemoveContainer" containerID="08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda"
Sep 11 19:37:23.949414 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:23.949397    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda\": container with ID starting with 08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda not found: ID does not exist" containerID="08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda"
Sep 11 19:37:23.949457 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.949423    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda"} err="failed to get container status \"08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda\": rpc error: code = NotFound desc = could not find container \"08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda\": container with ID starting with 08508b0f49ecaceeade64eb0147f20f98602e9d677f4b85be460b7f756326fda not found: ID does not exist"
Sep 11 19:37:23.960027 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.959999    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"]
Sep 11 19:37:23.963175 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:23.963150    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-472add/explicit-model-472add-kserve-66c997ffd9-dx2rr"]
Sep 11 19:37:24.547380 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:24.547337    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40525483-a300-487a-8f01-633b4e04be7a" path="/var/lib/kubelet/pods/40525483-a300-487a-8f01-633b4e04be7a/volumes"
Sep 11 19:37:27.016649 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:27.016610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:27.735385 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:27.735346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:28.415776 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:28.415744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:29.213865 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:29.213832    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:29.861255 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:29.861216    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:30.655148 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:30.655112    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:30.795143 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:30.795106    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:31.538257 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:31.538216    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7798-1e8a22"]
Sep 11 19:37:31.538744 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:31.538696    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40525483-a300-487a-8f01-633b4e04be7a" containerName="main"
Sep 11 19:37:31.538744 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:31.538709    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="40525483-a300-487a-8f01-633b4e04be7a" containerName="main"
Sep 11 19:37:31.538866 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:31.538781    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="40525483-a300-487a-8f01-633b4e04be7a" containerName="main"
Sep 11 19:37:31.543229 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:31.543199    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-1e8a22"
Sep 11 19:37:31.545702 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:31.545681    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-v69wf\""
Sep 11 19:37:31.547116 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:31.547076    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7798-1e8a22"]
Sep 11 19:37:31.632951 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:31.632920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxldv\" (UniqueName: \"kubernetes.io/projected/c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc-kube-api-access-zxldv\") pod \"test-curl-7798-1e8a22\" (UID: \"c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc\") " pod="opendatahub/test-curl-7798-1e8a22"
Sep 11 19:37:31.734531 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:31.734485    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxldv\" (UniqueName: \"kubernetes.io/projected/c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc-kube-api-access-zxldv\") pod \"test-curl-7798-1e8a22\" (UID: \"c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc\") " pod="opendatahub/test-curl-7798-1e8a22"
Sep 11 19:37:31.742208 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:31.742180    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxldv\" (UniqueName: \"kubernetes.io/projected/c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc-kube-api-access-zxldv\") pod \"test-curl-7798-1e8a22\" (UID: \"c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc\") " pod="opendatahub/test-curl-7798-1e8a22"
Sep 11 19:37:31.855986 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:31.855946    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-1e8a22"
Sep 11 19:37:32.125443 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:32.125384    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7798-1e8a22"]
Sep 11 19:37:32.127831 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:37:32.127788    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc90a2cc5_275c_4c35_8afe_75a0cb2a2ffc.slice/crio-6475173835d393256be0b33a70a0311d575326392243a37f8b3240bfe159c024 WatchSource:0}: Error finding container 6475173835d393256be0b33a70a0311d575326392243a37f8b3240bfe159c024: Status 404 returned error can't find the container with id 6475173835d393256be0b33a70a0311d575326392243a37f8b3240bfe159c024
Sep 11 19:37:32.984912 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:32.984870    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7798-1e8a22" event={"ID":"c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc","Type":"ContainerStarted","Data":"9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d"}
Sep 11 19:37:32.984912 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:32.984920    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7798-1e8a22" event={"ID":"c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc","Type":"ContainerStarted","Data":"6475173835d393256be0b33a70a0311d575326392243a37f8b3240bfe159c024"}
Sep 11 19:37:33.000219 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.000170    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7798-1e8a22" podStartSLOduration=2.000154071 podStartE2EDuration="2.000154071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:37:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:37:32.999618908 +0000 UTC m=+1156.928243810" watchObservedRunningTime="2026-09-11 19:37:33.000154071 +0000 UTC m=+1156.928778966"
Sep 11 19:37:33.575462 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.575421    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7798-1e8a22"]
Sep 11 19:37:33.579626 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.579595    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7798-1e8a22"]
Sep 11 19:37:33.689300 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.689207    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7798-3b0feb"]
Sep 11 19:37:33.689720 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.689699    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc" containerName="test-curl-7798-1e8a22"
Sep 11 19:37:33.689720 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.689719    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc" containerName="test-curl-7798-1e8a22"
Sep 11 19:37:33.689880 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.689792    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc" containerName="test-curl-7798-1e8a22"
Sep 11 19:37:33.693071 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.693051    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-3b0feb"
Sep 11 19:37:33.698956 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.698932    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7798-3b0feb"]
Sep 11 19:37:33.752596 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.752554    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c42v5\" (UniqueName: \"kubernetes.io/projected/261bb1a7-d285-4e6d-b65a-df4edcc44342-kube-api-access-c42v5\") pod \"test-curl-7798-3b0feb\" (UID: \"261bb1a7-d285-4e6d-b65a-df4edcc44342\") " pod="opendatahub/test-curl-7798-3b0feb"
Sep 11 19:37:33.853129 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.853085    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c42v5\" (UniqueName: \"kubernetes.io/projected/261bb1a7-d285-4e6d-b65a-df4edcc44342-kube-api-access-c42v5\") pod \"test-curl-7798-3b0feb\" (UID: \"261bb1a7-d285-4e6d-b65a-df4edcc44342\") " pod="opendatahub/test-curl-7798-3b0feb"
Sep 11 19:37:33.862652 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:33.862622    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c42v5\" (UniqueName: \"kubernetes.io/projected/261bb1a7-d285-4e6d-b65a-df4edcc44342-kube-api-access-c42v5\") pod \"test-curl-7798-3b0feb\" (UID: \"261bb1a7-d285-4e6d-b65a-df4edcc44342\") " pod="opendatahub/test-curl-7798-3b0feb"
Sep 11 19:37:33.989641 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:33.989535    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7798-1e8a22" container="test-curl-7798-1e8a22"
Sep 11 19:37:34.003968 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:34.003940    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-3b0feb"
Sep 11 19:37:34.274513 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:34.274488    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7798-3b0feb"]
Sep 11 19:37:34.276830 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:37:34.276803    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod261bb1a7_d285_4e6d_b65a_df4edcc44342.slice/crio-c17eb208635891503b8850d2ca692a3f62319d118d0cc65f4fdbfb3b1217c86f WatchSource:0}: Error finding container c17eb208635891503b8850d2ca692a3f62319d118d0cc65f4fdbfb3b1217c86f: Status 404 returned error can't find the container with id c17eb208635891503b8850d2ca692a3f62319d118d0cc65f4fdbfb3b1217c86f
Sep 11 19:37:34.995994 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:34.995952    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7798-3b0feb" event={"ID":"261bb1a7-d285-4e6d-b65a-df4edcc44342","Type":"ContainerStarted","Data":"5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753"}
Sep 11 19:37:34.996513 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:34.995993    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7798-1e8a22" podUID="c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc" containerName="test-curl-7798-1e8a22" containerID="cri-o://9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d" gracePeriod=2
Sep 11 19:37:34.996513 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:34.996006    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7798-3b0feb" event={"ID":"261bb1a7-d285-4e6d-b65a-df4edcc44342","Type":"ContainerStarted","Data":"c17eb208635891503b8850d2ca692a3f62319d118d0cc65f4fdbfb3b1217c86f"}
Sep 11 19:37:35.010976 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.010925    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7798-3b0feb" podStartSLOduration=2.010907542 podStartE2EDuration="2.010907542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:37:35.009728157 +0000 UTC m=+1158.938353058" watchObservedRunningTime="2026-09-11 19:37:35.010907542 +0000 UTC m=+1158.939532439"
Sep 11 19:37:35.572918 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.572878    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7798-3b0feb"]
Sep 11 19:37:35.577765 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.577737    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7798-3b0feb"]
Sep 11 19:37:35.692723 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.692630    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7798-9ff06a"]
Sep 11 19:37:35.693146 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.693126    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261bb1a7-d285-4e6d-b65a-df4edcc44342" containerName="test-curl-7798-3b0feb"
Sep 11 19:37:35.693231 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.693149    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="261bb1a7-d285-4e6d-b65a-df4edcc44342" containerName="test-curl-7798-3b0feb"
Sep 11 19:37:35.693335 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.693266    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="261bb1a7-d285-4e6d-b65a-df4edcc44342" containerName="test-curl-7798-3b0feb"
Sep 11 19:37:35.696500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.696479    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-9ff06a"
Sep 11 19:37:35.701506 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.701485    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7798-9ff06a"]
Sep 11 19:37:35.770543 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.770510    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x8d4\" (UniqueName: \"kubernetes.io/projected/044c340c-1d62-4cb2-a51a-83f53f722c59-kube-api-access-5x8d4\") pod \"test-curl-7798-9ff06a\" (UID: \"044c340c-1d62-4cb2-a51a-83f53f722c59\") " pod="opendatahub/test-curl-7798-9ff06a"
Sep 11 19:37:35.871349 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.871315    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x8d4\" (UniqueName: \"kubernetes.io/projected/044c340c-1d62-4cb2-a51a-83f53f722c59-kube-api-access-5x8d4\") pod \"test-curl-7798-9ff06a\" (UID: \"044c340c-1d62-4cb2-a51a-83f53f722c59\") " pod="opendatahub/test-curl-7798-9ff06a"
Sep 11 19:37:35.879389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:35.879356    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x8d4\" (UniqueName: \"kubernetes.io/projected/044c340c-1d62-4cb2-a51a-83f53f722c59-kube-api-access-5x8d4\") pod \"test-curl-7798-9ff06a\" (UID: \"044c340c-1d62-4cb2-a51a-83f53f722c59\") " pod="opendatahub/test-curl-7798-9ff06a"
Sep 11 19:37:36.000808 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:36.000734    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7798-3b0feb" container="test-curl-7798-3b0feb"
Sep 11 19:37:36.006669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:36.006651    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-9ff06a"
Sep 11 19:37:36.275210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:36.275182    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7798-9ff06a"]
Sep 11 19:37:36.277281 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:37:36.277233    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod044c340c_1d62_4cb2_a51a_83f53f722c59.slice/crio-b5169eac5dd76ef89de31fbcf437ed005fc3fc8a790c8126c8e7960937cd06ea WatchSource:0}: Error finding container b5169eac5dd76ef89de31fbcf437ed005fc3fc8a790c8126c8e7960937cd06ea: Status 404 returned error can't find the container with id b5169eac5dd76ef89de31fbcf437ed005fc3fc8a790c8126c8e7960937cd06ea
Sep 11 19:37:36.735307 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:36.735268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:37.015316 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.015284    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7798-9ff06a" event={"ID":"044c340c-1d62-4cb2-a51a-83f53f722c59","Type":"ContainerStarted","Data":"fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28"}
Sep 11 19:37:37.015674 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.015334    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7798-9ff06a" event={"ID":"044c340c-1d62-4cb2-a51a-83f53f722c59","Type":"ContainerStarted","Data":"b5169eac5dd76ef89de31fbcf437ed005fc3fc8a790c8126c8e7960937cd06ea"}
Sep 11 19:37:37.015674 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.015375    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7798-3b0feb" podUID="261bb1a7-d285-4e6d-b65a-df4edcc44342" containerName="test-curl-7798-3b0feb" containerID="cri-o://5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753" gracePeriod=2
Sep 11 19:37:37.029639 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.029587    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7798-9ff06a" podStartSLOduration=2.029571507 podStartE2EDuration="2.029571507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:37:37.028609145 +0000 UTC m=+1160.957234047" watchObservedRunningTime="2026-09-11 19:37:37.029571507 +0000 UTC m=+1160.958196405"
Sep 11 19:37:37.182508 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.182487    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-1e8a22"
Sep 11 19:37:37.284365 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.284284    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxldv\" (UniqueName: \"kubernetes.io/projected/c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc-kube-api-access-zxldv\") pod \"c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc\" (UID: \"c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc\") "
Sep 11 19:37:37.286736 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.286703    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc-kube-api-access-zxldv" (OuterVolumeSpecName: "kube-api-access-zxldv") pod "c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc" (UID: "c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc"). InnerVolumeSpecName "kube-api-access-zxldv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:37:37.385061 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.385033    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxldv\" (UniqueName: \"kubernetes.io/projected/c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc-kube-api-access-zxldv\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:37.588829 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.588717    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7798-9ff06a"]
Sep 11 19:37:37.594474 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.594449    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7798-9ff06a"]
Sep 11 19:37:37.708865 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.708830    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7798-c9e3af"]
Sep 11 19:37:37.709296 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.709282    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="044c340c-1d62-4cb2-a51a-83f53f722c59" containerName="test-curl-7798-9ff06a"
Sep 11 19:37:37.709356 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.709298    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="044c340c-1d62-4cb2-a51a-83f53f722c59" containerName="test-curl-7798-9ff06a"
Sep 11 19:37:37.709401 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.709364    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="044c340c-1d62-4cb2-a51a-83f53f722c59" containerName="test-curl-7798-9ff06a"
Sep 11 19:37:37.712548 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.712528    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-c9e3af"
Sep 11 19:37:37.716145 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.716120    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7798-c9e3af"]
Sep 11 19:37:37.788897 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.788859    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9hqz\" (UniqueName: \"kubernetes.io/projected/5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d-kube-api-access-q9hqz\") pod \"test-curl-7798-c9e3af\" (UID: \"5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d\") " pod="opendatahub/test-curl-7798-c9e3af"
Sep 11 19:37:37.890202 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.890169    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9hqz\" (UniqueName: \"kubernetes.io/projected/5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d-kube-api-access-q9hqz\") pod \"test-curl-7798-c9e3af\" (UID: \"5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d\") " pod="opendatahub/test-curl-7798-c9e3af"
Sep 11 19:37:37.898430 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:37.898398    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9hqz\" (UniqueName: \"kubernetes.io/projected/5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d-kube-api-access-q9hqz\") pod \"test-curl-7798-c9e3af\" (UID: \"5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d\") " pod="opendatahub/test-curl-7798-c9e3af"
Sep 11 19:37:38.021996 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:38.021962    2583 generic.go:356] "Generic (PLEG): container finished" podID="c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc" containerID="9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d" exitCode=137
Sep 11 19:37:38.022440 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:38.022004    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-1e8a22"
Sep 11 19:37:38.022440 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:38.022054    2583 scope.go:117] "RemoveContainer" containerID="9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d"
Sep 11 19:37:38.022440 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:38.022372    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7798-9ff06a" container="test-curl-7798-9ff06a"
Sep 11 19:37:38.029716 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:38.029695    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-c9e3af"
Sep 11 19:37:38.031208 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:38.031193    2583 scope.go:117] "RemoveContainer" containerID="9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d"
Sep 11 19:37:38.031517 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:38.031499    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d\": container with ID starting with 9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d not found: ID does not exist" containerID="9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d"
Sep 11 19:37:38.031573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:38.031526    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d"} err="failed to get container status \"9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d\": rpc error: code = NotFound desc = could not find container \"9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d\": container with ID starting with 9d84a8c109bbbd6f2a32e4e2a9eca8b468017a4dceea1ca12461e5210c66cb8d not found: ID does not exist"
Sep 11 19:37:38.032837 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:38.032812    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7798-1e8a22\" is forbidden: User \"system:node:ip-10-0-135-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-147.ec2.internal' and this object" podUID="c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc" pod="opendatahub/test-curl-7798-1e8a22"
Sep 11 19:37:38.298449 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:38.298423    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7798-c9e3af"]
Sep 11 19:37:38.300976 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:37:38.300950    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fb57ce8_efa4_45d1_884d_4dc14d5a5b2d.slice/crio-7c265f4e585dd87c797d4a8f55c04557519dbd52229db37976ea7d879e700cbe WatchSource:0}: Error finding container 7c265f4e585dd87c797d4a8f55c04557519dbd52229db37976ea7d879e700cbe: Status 404 returned error can't find the container with id 7c265f4e585dd87c797d4a8f55c04557519dbd52229db37976ea7d879e700cbe
Sep 11 19:37:38.546312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:38.546279    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc" path="/var/lib/kubelet/pods/c90a2cc5-275c-4c35-8afe-75a0cb2a2ffc/volumes"
Sep 11 19:37:39.029199 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:39.029168    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7798-c9e3af" event={"ID":"5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d","Type":"ContainerStarted","Data":"99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3"}
Sep 11 19:37:39.029684 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:39.029303    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7798-c9e3af" event={"ID":"5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d","Type":"ContainerStarted","Data":"7c265f4e585dd87c797d4a8f55c04557519dbd52229db37976ea7d879e700cbe"}
Sep 11 19:37:39.030360 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:39.030336    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7798-9ff06a" podUID="044c340c-1d62-4cb2-a51a-83f53f722c59" containerName="test-curl-7798-9ff06a" containerID="cri-o://fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28" gracePeriod=2
Sep 11 19:37:39.043429 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:39.043384    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7798-c9e3af" podStartSLOduration=2.043369268 podStartE2EDuration="2.043369268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:37:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:37:39.041717839 +0000 UTC m=+1162.970342738" watchObservedRunningTime="2026-09-11 19:37:39.043369268 +0000 UTC m=+1162.971994216"
Sep 11 19:37:39.212527 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:39.212502    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-3b0feb"
Sep 11 19:37:39.303099 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:39.303018    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c42v5\" (UniqueName: \"kubernetes.io/projected/261bb1a7-d285-4e6d-b65a-df4edcc44342-kube-api-access-c42v5\") pod \"261bb1a7-d285-4e6d-b65a-df4edcc44342\" (UID: \"261bb1a7-d285-4e6d-b65a-df4edcc44342\") "
Sep 11 19:37:39.305316 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:39.305284    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261bb1a7-d285-4e6d-b65a-df4edcc44342-kube-api-access-c42v5" (OuterVolumeSpecName: "kube-api-access-c42v5") pod "261bb1a7-d285-4e6d-b65a-df4edcc44342" (UID: "261bb1a7-d285-4e6d-b65a-df4edcc44342"). InnerVolumeSpecName "kube-api-access-c42v5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:37:39.403722 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:39.403694    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c42v5\" (UniqueName: \"kubernetes.io/projected/261bb1a7-d285-4e6d-b65a-df4edcc44342-kube-api-access-c42v5\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:39.603390 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:39.603356    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7798-c9e3af"]
Sep 11 19:37:39.607501 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:39.607475    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7798-c9e3af"]
Sep 11 19:37:40.037038 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:40.036949    2583 generic.go:356] "Generic (PLEG): container finished" podID="261bb1a7-d285-4e6d-b65a-df4edcc44342" containerID="5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753" exitCode=137
Sep 11 19:37:40.037038 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:40.037010    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-3b0feb"
Sep 11 19:37:40.037550 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:40.037045    2583 scope.go:117] "RemoveContainer" containerID="5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753"
Sep 11 19:37:40.037550 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:40.037493    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7798-c9e3af" container="test-curl-7798-c9e3af"
Sep 11 19:37:40.046171 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:40.046154    2583 scope.go:117] "RemoveContainer" containerID="5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753"
Sep 11 19:37:40.046450 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:40.046432    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753\": container with ID starting with 5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753 not found: ID does not exist" containerID="5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753"
Sep 11 19:37:40.046507 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:40.046460    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753"} err="failed to get container status \"5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753\": rpc error: code = NotFound desc = could not find container \"5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753\": container with ID starting with 5b3d0cf0481cf58bcdfb1f3c097cce9cd34d34e21bd18bc4dce5898732eb1753 not found: ID does not exist"
Sep 11 19:37:40.046803 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:40.046786    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7798-3b0feb\" is forbidden: User \"system:node:ip-10-0-135-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-147.ec2.internal' and this object" podUID="261bb1a7-d285-4e6d-b65a-df4edcc44342" pod="opendatahub/test-curl-7798-3b0feb"
Sep 11 19:37:40.194775 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:40.194728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:40.510395 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:40.510366    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:40.546199 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:40.546167    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="261bb1a7-d285-4e6d-b65a-df4edcc44342" path="/var/lib/kubelet/pods/261bb1a7-d285-4e6d-b65a-df4edcc44342/volumes"
Sep 11 19:37:41.038170 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:41.038025    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:41.043656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:41.043619    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7798-c9e3af" podUID="5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d" containerName="test-curl-7798-c9e3af" containerID="cri-o://99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3" gracePeriod=2
Sep 11 19:37:41.181362 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:41.181337    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-9ff06a"
Sep 11 19:37:41.219470 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:41.219443    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5x8d4\" (UniqueName: \"kubernetes.io/projected/044c340c-1d62-4cb2-a51a-83f53f722c59-kube-api-access-5x8d4\") pod \"044c340c-1d62-4cb2-a51a-83f53f722c59\" (UID: \"044c340c-1d62-4cb2-a51a-83f53f722c59\") "
Sep 11 19:37:41.221586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:41.221562    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/044c340c-1d62-4cb2-a51a-83f53f722c59-kube-api-access-5x8d4" (OuterVolumeSpecName: "kube-api-access-5x8d4") pod "044c340c-1d62-4cb2-a51a-83f53f722c59" (UID: "044c340c-1d62-4cb2-a51a-83f53f722c59"). InnerVolumeSpecName "kube-api-access-5x8d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:37:41.320320 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:41.320209    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x8d4\" (UniqueName: \"kubernetes.io/projected/044c340c-1d62-4cb2-a51a-83f53f722c59-kube-api-access-5x8d4\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:42.049621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:42.049576    2583 generic.go:356] "Generic (PLEG): container finished" podID="044c340c-1d62-4cb2-a51a-83f53f722c59" containerID="fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28" exitCode=137
Sep 11 19:37:42.050011 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:42.049625    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-9ff06a"
Sep 11 19:37:42.050011 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:42.049633    2583 scope.go:117] "RemoveContainer" containerID="fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28"
Sep 11 19:37:42.058960 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:42.058942    2583 scope.go:117] "RemoveContainer" containerID="fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28"
Sep 11 19:37:42.059226 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:42.059204    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7798-9ff06a\" is forbidden: User \"system:node:ip-10-0-135-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-147.ec2.internal' and this object" podUID="044c340c-1d62-4cb2-a51a-83f53f722c59" pod="opendatahub/test-curl-7798-9ff06a"
Sep 11 19:37:42.059312 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:42.059230    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28\": container with ID starting with fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28 not found: ID does not exist" containerID="fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28"
Sep 11 19:37:42.059312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:42.059283    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28"} err="failed to get container status \"fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28\": rpc error: code = NotFound desc = could not find container \"fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28\": container with ID starting with fe7ba90aa2a50fe7379ed5fb8d5142e29ad9442ad7fb0ec533b8d03074581a28 not found: ID does not exist"
Sep 11 19:37:42.547060 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:42.547024    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="044c340c-1d62-4cb2-a51a-83f53f722c59" path="/var/lib/kubelet/pods/044c340c-1d62-4cb2-a51a-83f53f722c59/volumes"
Sep 11 19:37:43.228374 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:43.228347    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-c9e3af"
Sep 11 19:37:43.332409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:43.332326    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:43.340066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:43.340037    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9hqz\" (UniqueName: \"kubernetes.io/projected/5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d-kube-api-access-q9hqz\") pod \"5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d\" (UID: \"5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d\") "
Sep 11 19:37:43.342342 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:43.342314    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d-kube-api-access-q9hqz" (OuterVolumeSpecName: "kube-api-access-q9hqz") pod "5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d" (UID: "5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d"). InnerVolumeSpecName "kube-api-access-q9hqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:37:43.440910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:43.440874    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9hqz\" (UniqueName: \"kubernetes.io/projected/5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d-kube-api-access-q9hqz\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:37:44.032627 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:44.032585    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:44.062709 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:44.062670    2583 generic.go:356] "Generic (PLEG): container finished" podID="5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d" containerID="99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3" exitCode=137
Sep 11 19:37:44.062888 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:44.062727    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7798-c9e3af"
Sep 11 19:37:44.062888 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:44.062734    2583 scope.go:117] "RemoveContainer" containerID="99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3"
Sep 11 19:37:44.072164 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:44.072145    2583 scope.go:117] "RemoveContainer" containerID="99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3"
Sep 11 19:37:44.072454 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:44.072432    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3\": container with ID starting with 99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3 not found: ID does not exist" containerID="99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3"
Sep 11 19:37:44.072536 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:44.072463    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3"} err="failed to get container status \"99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3\": rpc error: code = NotFound desc = could not find container \"99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3\": container with ID starting with 99404f9a07520ae96ceb4dc405f74a199c061673832ebf546a3ad3f1ef986ee3 not found: ID does not exist"
Sep 11 19:37:44.072801 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:44.072778    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7798-c9e3af\" is forbidden: User \"system:node:ip-10-0-135-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-147.ec2.internal' and this object" podUID="5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d" pod="opendatahub/test-curl-7798-c9e3af"
Sep 11 19:37:44.123316 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:44.123287    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:44.547772 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:44.547742    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d" path="/var/lib/kubelet/pods/5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d/volumes"
Sep 11 19:37:44.920138 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:44.920098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:45.630634 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:45.630601    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:45.844818 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:45.844781    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:45.944973 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:45.944892    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:46.268014 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:46.267924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:46.576755 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:46.576674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:46.677444 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:46.677407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:47.069929 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:47.069896    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:47.488873 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:47.488838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:48.229336 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.229300    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"]
Sep 11 19:37:48.229738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.229723    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d" containerName="test-curl-7798-c9e3af"
Sep 11 19:37:48.229796 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.229740    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d" containerName="test-curl-7798-c9e3af"
Sep 11 19:37:48.229834 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.229806    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb57ce8-efa4-45d1-884d-4dc14d5a5b2d" containerName="test-curl-7798-c9e3af"
Sep 11 19:37:48.234199 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.234183    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.236911 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.236888    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-55586842-serving-cert\""
Sep 11 19:37:48.245783 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.245760    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"]
Sep 11 19:37:48.278268 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.278209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:48.286621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.286586    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-openshift-service-ca\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.286761 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.286713    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-maas-api-tls\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.286826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.286782    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-metrics-certs\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.286826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.286800    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc77t\" (UniqueName: \"kubernetes.io/projected/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-kube-api-access-cc77t\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.372762 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.372719    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:48.388033 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.388006    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc77t\" (UniqueName: \"kubernetes.io/projected/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-kube-api-access-cc77t\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.388150 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.388052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-openshift-service-ca\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.388150 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.388112    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-maas-api-tls\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.388225 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.388150    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-metrics-certs\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.388301 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:48.388227    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-55586842-serving-cert: secret "maas-api-e2e-mt-55586842-serving-cert" not found
Sep 11 19:37:48.388343 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:37:48.388324    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-maas-api-tls podName:0770c927-94aa-4ff7-b1e3-b44e28cdcd8c nodeName:}" failed. No retries permitted until 2026-09-11 19:37:48.888304473 +0000 UTC m=+1172.816929364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-maas-api-tls") pod "maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws" (UID: "0770c927-94aa-4ff7-b1e3-b44e28cdcd8c") : secret "maas-api-e2e-mt-55586842-serving-cert" not found
Sep 11 19:37:48.388765 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.388745    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-openshift-service-ca\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.390673 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.390654    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-metrics-certs\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.398278 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.398228    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc77t\" (UniqueName: \"kubernetes.io/projected/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-kube-api-access-cc77t\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.525725 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.525635    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:48.628742 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.628704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:48.891831 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.891798    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-maas-api-tls\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:48.894574 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:48.894551    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-maas-api-tls\") pod \"maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:49.033600 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.033564    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:49.127753 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.127721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:37:49.143340 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.143230    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"]
Sep 11 19:37:49.146849 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.146825    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:49.147047 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.147030    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:37:49.149586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.149562    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-55586842\""
Sep 11 19:37:49.149735 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.149562    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-55586842-dockercfg-66p8v\""
Sep 11 19:37:49.161085 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.161057    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"]
Sep 11 19:37:49.176669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.176639    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"]
Sep 11 19:37:49.181346 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.181323    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:37:49.193802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.193767    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2195476-40b4-4785-b27b-8fe3ece5f799-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh\" (UID: \"b2195476-40b4-4785-b27b-8fe3ece5f799\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:37:49.194000 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.193842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds9vj\" (UniqueName: \"kubernetes.io/projected/b2195476-40b4-4785-b27b-8fe3ece5f799-kube-api-access-ds9vj\") pod \"payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh\" (UID: \"b2195476-40b4-4785-b27b-8fe3ece5f799\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:37:49.196431 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.196405    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"]
Sep 11 19:37:49.295330 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.295291    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgcc2\" (UniqueName: \"kubernetes.io/projected/c879b361-9e26-4a3d-ac39-33dc7ea1e800-kube-api-access-hgcc2\") pod \"payload-processing-e2e-mt-55586842-84485f948f-xnx67\" (UID: \"c879b361-9e26-4a3d-ac39-33dc7ea1e800\") " pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:37:49.295784 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.295349    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2195476-40b4-4785-b27b-8fe3ece5f799-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh\" (UID: \"b2195476-40b4-4785-b27b-8fe3ece5f799\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:37:49.295784 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.295383    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c879b361-9e26-4a3d-ac39-33dc7ea1e800-plugins-config-volume\") pod \"payload-processing-e2e-mt-55586842-84485f948f-xnx67\" (UID: \"c879b361-9e26-4a3d-ac39-33dc7ea1e800\") " pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:37:49.295784 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.295435    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds9vj\" (UniqueName: \"kubernetes.io/projected/b2195476-40b4-4785-b27b-8fe3ece5f799-kube-api-access-ds9vj\") pod \"payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh\" (UID: \"b2195476-40b4-4785-b27b-8fe3ece5f799\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:37:49.296052 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.296030    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2195476-40b4-4785-b27b-8fe3ece5f799-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh\" (UID: \"b2195476-40b4-4785-b27b-8fe3ece5f799\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:37:49.304071 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.304038    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds9vj\" (UniqueName: \"kubernetes.io/projected/b2195476-40b4-4785-b27b-8fe3ece5f799-kube-api-access-ds9vj\") pod \"payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh\" (UID: \"b2195476-40b4-4785-b27b-8fe3ece5f799\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:37:49.396817 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.396723    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgcc2\" (UniqueName: \"kubernetes.io/projected/c879b361-9e26-4a3d-ac39-33dc7ea1e800-kube-api-access-hgcc2\") pod \"payload-processing-e2e-mt-55586842-84485f948f-xnx67\" (UID: \"c879b361-9e26-4a3d-ac39-33dc7ea1e800\") " pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:37:49.396817 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.396780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c879b361-9e26-4a3d-ac39-33dc7ea1e800-plugins-config-volume\") pod \"payload-processing-e2e-mt-55586842-84485f948f-xnx67\" (UID: \"c879b361-9e26-4a3d-ac39-33dc7ea1e800\") " pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:37:49.397530 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.397507    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c879b361-9e26-4a3d-ac39-33dc7ea1e800-plugins-config-volume\") pod \"payload-processing-e2e-mt-55586842-84485f948f-xnx67\" (UID: \"c879b361-9e26-4a3d-ac39-33dc7ea1e800\") " pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:37:49.404847 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.404819    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgcc2\" (UniqueName: \"kubernetes.io/projected/c879b361-9e26-4a3d-ac39-33dc7ea1e800-kube-api-access-hgcc2\") pod \"payload-processing-e2e-mt-55586842-84485f948f-xnx67\" (UID: \"c879b361-9e26-4a3d-ac39-33dc7ea1e800\") " pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:37:49.432092 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.432069    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"]
Sep 11 19:37:49.433942 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:37:49.433910    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0770c927_94aa_4ff7_b1e3_b44e28cdcd8c.slice/crio-868a0ce9f5ebd12029e758de7718ced458a7ce2aecc277b8d6699acc36e97599 WatchSource:0}: Error finding container 868a0ce9f5ebd12029e758de7718ced458a7ce2aecc277b8d6699acc36e97599: Status 404 returned error can't find the container with id 868a0ce9f5ebd12029e758de7718ced458a7ce2aecc277b8d6699acc36e97599
Sep 11 19:37:49.478180 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.478147    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:37:49.493271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.493227    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:37:49.780937 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.780912    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"]
Sep 11 19:37:49.782482 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:37:49.782458    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2195476_40b4_4785_b27b_8fe3ece5f799.slice/crio-7f20a0d48de20bcc4746f04ef231fe9397b380b5da00e811b700ba42dd614e73 WatchSource:0}: Error finding container 7f20a0d48de20bcc4746f04ef231fe9397b380b5da00e811b700ba42dd614e73: Status 404 returned error can't find the container with id 7f20a0d48de20bcc4746f04ef231fe9397b380b5da00e811b700ba42dd614e73
Sep 11 19:37:49.798312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:49.798286    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"]
Sep 11 19:37:49.800224 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:37:49.800190    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc879b361_9e26_4a3d_ac39_33dc7ea1e800.slice/crio-2467070b49167458933d9cc32a33d66287e496b429fa8610fed41c4032001c7a WatchSource:0}: Error finding container 2467070b49167458933d9cc32a33d66287e496b429fa8610fed41c4032001c7a: Status 404 returned error can't find the container with id 2467070b49167458933d9cc32a33d66287e496b429fa8610fed41c4032001c7a
Sep 11 19:37:50.099092 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.099058    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh" event={"ID":"b2195476-40b4-4785-b27b-8fe3ece5f799","Type":"ContainerStarted","Data":"000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2"}
Sep 11 19:37:50.099308 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.099125    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh" event={"ID":"b2195476-40b4-4785-b27b-8fe3ece5f799","Type":"ContainerStarted","Data":"7f20a0d48de20bcc4746f04ef231fe9397b380b5da00e811b700ba42dd614e73"}
Sep 11 19:37:50.099308 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.099270    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:37:50.101135 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.101108    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67" event={"ID":"c879b361-9e26-4a3d-ac39-33dc7ea1e800","Type":"ContainerStarted","Data":"3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b"}
Sep 11 19:37:50.101349 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.101165    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67" event={"ID":"c879b361-9e26-4a3d-ac39-33dc7ea1e800","Type":"ContainerStarted","Data":"2467070b49167458933d9cc32a33d66287e496b429fa8610fed41c4032001c7a"}
Sep 11 19:37:50.101349 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.101206    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:37:50.103184 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.103155    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws" event={"ID":"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c","Type":"ContainerStarted","Data":"7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e"}
Sep 11 19:37:50.103866 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.103204    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws" event={"ID":"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c","Type":"ContainerStarted","Data":"868a0ce9f5ebd12029e758de7718ced458a7ce2aecc277b8d6699acc36e97599"}
Sep 11 19:37:50.103866 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.103229    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:37:50.116024 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.115975    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh" podStartSLOduration=1.115961421 podStartE2EDuration="1.115961421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:37:50.114077722 +0000 UTC m=+1174.042702622" watchObservedRunningTime="2026-09-11 19:37:50.115961421 +0000 UTC m=+1174.044586348"
Sep 11 19:37:50.131109 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.131057    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67" podStartSLOduration=1.13104036 podStartE2EDuration="1.13104036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:37:50.129071272 +0000 UTC m=+1174.057696170" watchObservedRunningTime="2026-09-11 19:37:50.13104036 +0000 UTC m=+1174.059665263"
Sep 11 19:37:50.146746 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:50.146683    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws" podStartSLOduration=1.734859579 podStartE2EDuration="2.146667919s" totalImagesPullingTime="411.80834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:37:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:37:50.145272115 +0000 UTC m=+1174.073897199" watchObservedRunningTime="2026-09-11 19:37:50.146667919 +0000 UTC m=+1174.075292818"
Sep 11 19:37:56.115492 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:37:56.115462    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:38:01.111976 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:01.111938    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:38:01.112517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:01.112285    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:38:11.875817 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:11.875764    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:12.520173 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:12.520113    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:13.338925 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:13.338887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:14.027905 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:14.027812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:14.123925 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:14.123887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:14.767600 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:14.767551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:15.540256 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:15.540214    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:16.285125 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:16.285089    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:22.803516 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.803476    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"]
Sep 11 19:38:22.808133 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.808104    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.810504 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.810481    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-967296\"/\"openshift-service-ca.crt\""
Sep 11 19:38:22.810650 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.810519    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-967296\"/\"kube-root-ca.crt\""
Sep 11 19:38:22.811368 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.811342    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-967296\"/\"default-dockercfg-26swk\""
Sep 11 19:38:22.811505 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.811372    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-967296\"/\"noten-model-967296-kserve-self-signed-certs\""
Sep 11 19:38:22.818474 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.818447    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"]
Sep 11 19:38:22.820573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.820492    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a93f63ac-6153-4c47-acce-61cb35522d73-tls-certs\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.820573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.820548    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-model-cache\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.821070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.820594    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-tmp-dir\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.821070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.820703    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-dshm\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.821070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.820763    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwfdq\" (UniqueName: \"kubernetes.io/projected/a93f63ac-6153-4c47-acce-61cb35522d73-kube-api-access-kwfdq\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.821070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.820815    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-home\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.837535 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.837504    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:22.921432 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.921396    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-tmp-dir\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.921651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.921454    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-dshm\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.921651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.921478    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfdq\" (UniqueName: \"kubernetes.io/projected/a93f63ac-6153-4c47-acce-61cb35522d73-kube-api-access-kwfdq\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.921651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.921504    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-home\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.921651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.921540    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a93f63ac-6153-4c47-acce-61cb35522d73-tls-certs\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.921651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.921563    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-model-cache\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.921910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.921831    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-tmp-dir\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.921950 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.921914    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-home\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.921988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.921945    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-model-cache\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.923914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.923896    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-dshm\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.924180 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.924160    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a93f63ac-6153-4c47-acce-61cb35522d73-tls-certs\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:22.929777 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:22.929751    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwfdq\" (UniqueName: \"kubernetes.io/projected/a93f63ac-6153-4c47-acce-61cb35522d73-kube-api-access-kwfdq\") pod \"noten-model-967296-kserve-678c9b6b59-4jrql\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") " pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:23.119954 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:23.119913    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:23.393760 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:23.393727    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"]
Sep 11 19:38:23.395291 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:38:23.395229    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda93f63ac_6153_4c47_acce_61cb35522d73.slice/crio-84fe51d3b49501155f7d6afc03f12b549b876b1b63064a14180deded15662392 WatchSource:0}: Error finding container 84fe51d3b49501155f7d6afc03f12b549b876b1b63064a14180deded15662392: Status 404 returned error can't find the container with id 84fe51d3b49501155f7d6afc03f12b549b876b1b63064a14180deded15662392
Sep 11 19:38:24.077532 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:24.077472    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:24.281560 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:24.281518    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql" event={"ID":"a93f63ac-6153-4c47-acce-61cb35522d73","Type":"ContainerStarted","Data":"3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5"}
Sep 11 19:38:24.281560 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:24.281567    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql" event={"ID":"a93f63ac-6153-4c47-acce-61cb35522d73","Type":"ContainerStarted","Data":"84fe51d3b49501155f7d6afc03f12b549b876b1b63064a14180deded15662392"}
Sep 11 19:38:24.299776 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:24.299726    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql" podStartSLOduration=2.299708782 podStartE2EDuration="2.299708782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:38:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:38:24.297129254 +0000 UTC m=+1208.225754152" watchObservedRunningTime="2026-09-11 19:38:24.299708782 +0000 UTC m=+1208.228333740"
Sep 11 19:38:25.519695 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:25.519664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:25.631299 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:25.631261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:33.120609 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:33.120574    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:33.123036 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:33.120676    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:33.133560 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:33.133536    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:33.337894 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:33.337866    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:38:42.738959 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:42.738927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:53.228836 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:53.228793    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:55.778807 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:55.778762    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"]
Sep 11 19:38:55.802929 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:55.802898    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:55.926480 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:55.926438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:56.148395 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:56.148360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:56.262539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:56.262505    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:56.553802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:56.553699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:57.357602 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:57.357561    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:58.146563 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:58.146532    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:58.708117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:58.708077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:58.844423 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:58.844389    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:38:59.505465 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:38:59.505431    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:00.352231 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:00.352192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:01.105066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:01.105035    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:01.296622 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:01.296586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:01.748115 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:01.748086    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:06.823878 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:06.823843    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"]
Sep 11 19:39:06.824406 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:06.824074    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m" podUID="77daedc1-6d88-4c54-a00a-d6039d9474c9" containerName="maas-api" containerID="cri-o://b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a" gracePeriod=30
Sep 11 19:39:06.850383 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:06.850348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:06.879876 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:06.879841    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"]
Sep 11 19:39:06.880123 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:06.880098    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2" podUID="32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf" containerName="payload-processing" containerID="cri-o://3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce" gracePeriod=30
Sep 11 19:39:06.886537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:06.885760    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"]
Sep 11 19:39:06.886537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:06.885994    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr" podUID="0d14a765-b470-4422-88d0-a7dd292f277f" containerName="payload-pre-processing" containerID="cri-o://18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed" gracePeriod=30
Sep 11 19:39:07.184606 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.184578    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:39:07.187762 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.187742    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:39:07.191683 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.191666    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:39:07.272386 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.272354    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2797c\" (UniqueName: \"kubernetes.io/projected/77daedc1-6d88-4c54-a00a-d6039d9474c9-kube-api-access-2797c\") pod \"77daedc1-6d88-4c54-a00a-d6039d9474c9\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") "
Sep 11 19:39:07.272586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.272408    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j684b\" (UniqueName: \"kubernetes.io/projected/0d14a765-b470-4422-88d0-a7dd292f277f-kube-api-access-j684b\") pod \"0d14a765-b470-4422-88d0-a7dd292f277f\" (UID: \"0d14a765-b470-4422-88d0-a7dd292f277f\") "
Sep 11 19:39:07.272586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.272463    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-plugins-config-volume\") pod \"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf\" (UID: \"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf\") "
Sep 11 19:39:07.272586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.272503    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsf9k\" (UniqueName: \"kubernetes.io/projected/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-kube-api-access-nsf9k\") pod \"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf\" (UID: \"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf\") "
Sep 11 19:39:07.272586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.272547    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-maas-api-tls\") pod \"77daedc1-6d88-4c54-a00a-d6039d9474c9\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") "
Sep 11 19:39:07.272586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.272568    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77daedc1-6d88-4c54-a00a-d6039d9474c9-openshift-service-ca\") pod \"77daedc1-6d88-4c54-a00a-d6039d9474c9\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") "
Sep 11 19:39:07.272840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.272619    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d14a765-b470-4422-88d0-a7dd292f277f-plugins-config-volume\") pod \"0d14a765-b470-4422-88d0-a7dd292f277f\" (UID: \"0d14a765-b470-4422-88d0-a7dd292f277f\") "
Sep 11 19:39:07.272840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.272676    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-metrics-certs\") pod \"77daedc1-6d88-4c54-a00a-d6039d9474c9\" (UID: \"77daedc1-6d88-4c54-a00a-d6039d9474c9\") "
Sep 11 19:39:07.273417 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.273043    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77daedc1-6d88-4c54-a00a-d6039d9474c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "77daedc1-6d88-4c54-a00a-d6039d9474c9" (UID: "77daedc1-6d88-4c54-a00a-d6039d9474c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:39:07.273688 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.273633    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d14a765-b470-4422-88d0-a7dd292f277f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0d14a765-b470-4422-88d0-a7dd292f277f" (UID: "0d14a765-b470-4422-88d0-a7dd292f277f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:39:07.273890 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.273710    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf" (UID: "32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:39:07.275893 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.275860    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "77daedc1-6d88-4c54-a00a-d6039d9474c9" (UID: "77daedc1-6d88-4c54-a00a-d6039d9474c9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:39:07.276010 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.275966    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d14a765-b470-4422-88d0-a7dd292f277f-kube-api-access-j684b" (OuterVolumeSpecName: "kube-api-access-j684b") pod "0d14a765-b470-4422-88d0-a7dd292f277f" (UID: "0d14a765-b470-4422-88d0-a7dd292f277f"). InnerVolumeSpecName "kube-api-access-j684b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:39:07.276414 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.276069    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "77daedc1-6d88-4c54-a00a-d6039d9474c9" (UID: "77daedc1-6d88-4c54-a00a-d6039d9474c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:39:07.276414 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.276171    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77daedc1-6d88-4c54-a00a-d6039d9474c9-kube-api-access-2797c" (OuterVolumeSpecName: "kube-api-access-2797c") pod "77daedc1-6d88-4c54-a00a-d6039d9474c9" (UID: "77daedc1-6d88-4c54-a00a-d6039d9474c9"). InnerVolumeSpecName "kube-api-access-2797c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:39:07.277541 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.277521    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-kube-api-access-nsf9k" (OuterVolumeSpecName: "kube-api-access-nsf9k") pod "32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf" (UID: "32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf"). InnerVolumeSpecName "kube-api-access-nsf9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:39:07.374409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.374231    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2797c\" (UniqueName: \"kubernetes.io/projected/77daedc1-6d88-4c54-a00a-d6039d9474c9-kube-api-access-2797c\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:07.374409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.374305    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j684b\" (UniqueName: \"kubernetes.io/projected/0d14a765-b470-4422-88d0-a7dd292f277f-kube-api-access-j684b\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:07.374409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.374318    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:07.374409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.374334    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsf9k\" (UniqueName: \"kubernetes.io/projected/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf-kube-api-access-nsf9k\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:07.374409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.374348    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:07.374409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.374360    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77daedc1-6d88-4c54-a00a-d6039d9474c9-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:07.374409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.374372    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d14a765-b470-4422-88d0-a7dd292f277f-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:07.374409 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.374384    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77daedc1-6d88-4c54-a00a-d6039d9474c9-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:07.500056 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.500010    2583 generic.go:356] "Generic (PLEG): container finished" podID="77daedc1-6d88-4c54-a00a-d6039d9474c9" containerID="b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a" exitCode=0
Sep 11 19:39:07.500222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.500062    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"
Sep 11 19:39:07.500222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.500106    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m" event={"ID":"77daedc1-6d88-4c54-a00a-d6039d9474c9","Type":"ContainerDied","Data":"b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a"}
Sep 11 19:39:07.500222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.500163    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m" event={"ID":"77daedc1-6d88-4c54-a00a-d6039d9474c9","Type":"ContainerDied","Data":"71437acf544e5f101c2c1d15baec3125769e75edd5c93d6d616d72cd2fbdd06b"}
Sep 11 19:39:07.500222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.500187    2583 scope.go:117] "RemoveContainer" containerID="b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a"
Sep 11 19:39:07.501657 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.501635    2583 generic.go:356] "Generic (PLEG): container finished" podID="32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf" containerID="3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce" exitCode=0
Sep 11 19:39:07.501745 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.501678    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"
Sep 11 19:39:07.501800 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.501758    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2" event={"ID":"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf","Type":"ContainerDied","Data":"3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce"}
Sep 11 19:39:07.501858 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.501804    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2" event={"ID":"32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf","Type":"ContainerDied","Data":"fbccd1d851652f5694de6b893257bc1f1053a15a507ec422793fb0456fb6a401"}
Sep 11 19:39:07.503378 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.503350    2583 generic.go:356] "Generic (PLEG): container finished" podID="0d14a765-b470-4422-88d0-a7dd292f277f" containerID="18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed" exitCode=0
Sep 11 19:39:07.503482 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.503397    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr" event={"ID":"0d14a765-b470-4422-88d0-a7dd292f277f","Type":"ContainerDied","Data":"18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed"}
Sep 11 19:39:07.503482 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.503440    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr" event={"ID":"0d14a765-b470-4422-88d0-a7dd292f277f","Type":"ContainerDied","Data":"d5013cf8dcda816679dcaa11fddc9d611f64d8b56e98617a1deaa8c61cbd12ac"}
Sep 11 19:39:07.503600 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.503500    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"
Sep 11 19:39:07.510832 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.510811    2583 scope.go:117] "RemoveContainer" containerID="b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a"
Sep 11 19:39:07.511159 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:39:07.511137    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a\": container with ID starting with b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a not found: ID does not exist" containerID="b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a"
Sep 11 19:39:07.511231 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.511167    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a"} err="failed to get container status \"b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a\": rpc error: code = NotFound desc = could not find container \"b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a\": container with ID starting with b2cdc9266b33389115c70fe39aa17c6d0aa047d9710b35dae63efa2a0576c92a not found: ID does not exist"
Sep 11 19:39:07.511231 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.511184    2583 scope.go:117] "RemoveContainer" containerID="3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce"
Sep 11 19:39:07.521031 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.521014    2583 scope.go:117] "RemoveContainer" containerID="3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce"
Sep 11 19:39:07.521403 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:39:07.521384    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce\": container with ID starting with 3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce not found: ID does not exist" containerID="3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce"
Sep 11 19:39:07.521451 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.521411    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce"} err="failed to get container status \"3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce\": rpc error: code = NotFound desc = could not find container \"3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce\": container with ID starting with 3f65810a85f0dfd80d862f69d1a138e1eaa79c9bc20951273bdbdb2578ac3bce not found: ID does not exist"
Sep 11 19:39:07.521451 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.521427    2583 scope.go:117] "RemoveContainer" containerID="18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed"
Sep 11 19:39:07.526057 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.526030    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"]
Sep 11 19:39:07.530148 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.530131    2583 scope.go:117] "RemoveContainer" containerID="18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed"
Sep 11 19:39:07.530501 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:39:07.530479    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed\": container with ID starting with 18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed not found: ID does not exist" containerID="18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed"
Sep 11 19:39:07.530601 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.530507    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed"} err="failed to get container status \"18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed\": rpc error: code = NotFound desc = could not find container \"18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed\": container with ID starting with 18d4c3bd8354b4f5f4073b43ce9d8845f76185e001b0f575ea43c0672c35a5ed not found: ID does not exist"
Sep 11 19:39:07.532159 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.532136    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a0610b-656d4dc8b5-bkk2m"]
Sep 11 19:39:07.543998 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.543975    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"]
Sep 11 19:39:07.549443 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.549422    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a0610b-54c97dcbb5-zddd2"]
Sep 11 19:39:07.559710 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.559687    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"]
Sep 11 19:39:07.563693 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:07.563672    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a0610b-69cb6cd6c8-4dhdr"]
Sep 11 19:39:08.052719 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:08.052678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:08.230869 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:08.230837    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:08.547174 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:08.547081    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d14a765-b470-4422-88d0-a7dd292f277f" path="/var/lib/kubelet/pods/0d14a765-b470-4422-88d0-a7dd292f277f/volumes"
Sep 11 19:39:08.547591 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:08.547570    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf" path="/var/lib/kubelet/pods/32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf/volumes"
Sep 11 19:39:08.547989 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:08.547966    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77daedc1-6d88-4c54-a00a-d6039d9474c9" path="/var/lib/kubelet/pods/77daedc1-6d88-4c54-a00a-d6039d9474c9/volumes"
Sep 11 19:39:09.046607 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:09.046571    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:09.731952 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:09.731914    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:10.392743 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:10.392708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:10.800797 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:10.800710    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql" podUID="a93f63ac-6153-4c47-acce-61cb35522d73" containerName="main" containerID="cri-o://3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5" gracePeriod=45
Sep 11 19:39:11.054090 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.054029    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:39:11.110516 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.110484    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-dshm\") pod \"a93f63ac-6153-4c47-acce-61cb35522d73\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") "
Sep 11 19:39:11.110715 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.110533    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-model-cache\") pod \"a93f63ac-6153-4c47-acce-61cb35522d73\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") "
Sep 11 19:39:11.110715 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.110551    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-tmp-dir\") pod \"a93f63ac-6153-4c47-acce-61cb35522d73\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") "
Sep 11 19:39:11.110715 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.110619    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwfdq\" (UniqueName: \"kubernetes.io/projected/a93f63ac-6153-4c47-acce-61cb35522d73-kube-api-access-kwfdq\") pod \"a93f63ac-6153-4c47-acce-61cb35522d73\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") "
Sep 11 19:39:11.110715 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.110654    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a93f63ac-6153-4c47-acce-61cb35522d73-tls-certs\") pod \"a93f63ac-6153-4c47-acce-61cb35522d73\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") "
Sep 11 19:39:11.110715 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.110689    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-home\") pod \"a93f63ac-6153-4c47-acce-61cb35522d73\" (UID: \"a93f63ac-6153-4c47-acce-61cb35522d73\") "
Sep 11 19:39:11.111071 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.110860    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a93f63ac-6153-4c47-acce-61cb35522d73" (UID: "a93f63ac-6153-4c47-acce-61cb35522d73"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:39:11.111071 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.111016    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-home" (OuterVolumeSpecName: "home") pod "a93f63ac-6153-4c47-acce-61cb35522d73" (UID: "a93f63ac-6153-4c47-acce-61cb35522d73"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:39:11.111071 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.111017    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-tmp-dir\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:11.111071 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.110821    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-model-cache" (OuterVolumeSpecName: "model-cache") pod "a93f63ac-6153-4c47-acce-61cb35522d73" (UID: "a93f63ac-6153-4c47-acce-61cb35522d73"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:39:11.112938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.112910    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-dshm" (OuterVolumeSpecName: "dshm") pod "a93f63ac-6153-4c47-acce-61cb35522d73" (UID: "a93f63ac-6153-4c47-acce-61cb35522d73"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:39:11.113047 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.112979    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a93f63ac-6153-4c47-acce-61cb35522d73-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a93f63ac-6153-4c47-acce-61cb35522d73" (UID: "a93f63ac-6153-4c47-acce-61cb35522d73"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:39:11.113047 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.113014    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a93f63ac-6153-4c47-acce-61cb35522d73-kube-api-access-kwfdq" (OuterVolumeSpecName: "kube-api-access-kwfdq") pod "a93f63ac-6153-4c47-acce-61cb35522d73" (UID: "a93f63ac-6153-4c47-acce-61cb35522d73"). InnerVolumeSpecName "kube-api-access-kwfdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:39:11.211772 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.211738    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-dshm\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:11.211772 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.211765    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-model-cache\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:11.211772 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.211774    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwfdq\" (UniqueName: \"kubernetes.io/projected/a93f63ac-6153-4c47-acce-61cb35522d73-kube-api-access-kwfdq\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:11.211772 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.211782    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a93f63ac-6153-4c47-acce-61cb35522d73-tls-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:11.212022 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.211790    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a93f63ac-6153-4c47-acce-61cb35522d73-home\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:11.252351 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.252320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:11.529171 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.529135    2583 generic.go:356] "Generic (PLEG): container finished" podID="a93f63ac-6153-4c47-acce-61cb35522d73" containerID="3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5" exitCode=0
Sep 11 19:39:11.529369 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.529183    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"
Sep 11 19:39:11.529369 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.529218    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql" event={"ID":"a93f63ac-6153-4c47-acce-61cb35522d73","Type":"ContainerDied","Data":"3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5"}
Sep 11 19:39:11.529369 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.529279    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql" event={"ID":"a93f63ac-6153-4c47-acce-61cb35522d73","Type":"ContainerDied","Data":"84fe51d3b49501155f7d6afc03f12b549b876b1b63064a14180deded15662392"}
Sep 11 19:39:11.529369 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.529298    2583 scope.go:117] "RemoveContainer" containerID="3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5"
Sep 11 19:39:11.539500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.539478    2583 scope.go:117] "RemoveContainer" containerID="3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5"
Sep 11 19:39:11.539829 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:39:11.539807    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5\": container with ID starting with 3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5 not found: ID does not exist" containerID="3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5"
Sep 11 19:39:11.539898 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.539843    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5"} err="failed to get container status \"3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5\": rpc error: code = NotFound desc = could not find container \"3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5\": container with ID starting with 3de4b791dd08534b5b0f85bfe1563b0e765e73d2ff294f4ee74e0bcebad253b5 not found: ID does not exist"
Sep 11 19:39:11.550028 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.550002    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"]
Sep 11 19:39:11.551824 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:11.551804    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-967296/noten-model-967296-kserve-678c9b6b59-4jrql"]
Sep 11 19:39:12.546623 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:12.546586    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a93f63ac-6153-4c47-acce-61cb35522d73" path="/var/lib/kubelet/pods/a93f63ac-6153-4c47-acce-61cb35522d73/volumes"
Sep 11 19:39:13.110461 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:13.110428    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:13.211981 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:13.211926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:13.360311 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:13.360281    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:14.442281 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:14.442211    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:15.073330 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:15.073271    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:15.310301 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:15.310268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:16.027864 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:16.027825    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:16.704066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:16.704031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:17.543527 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:17.543493    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:18.324656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:18.324624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:24.210924 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.210887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:24.237952 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.237921    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"]
Sep 11 19:39:24.238441 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.238407    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt" podUID="a1894a02-4494-4cc0-970d-ba97f2309187" containerName="payload-processing" containerID="cri-o://396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b" gracePeriod=30
Sep 11 19:39:24.312515 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.312484    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:24.411673 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.411646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:24.506778 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.506754    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:39:24.512146 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.512124    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:24.596056 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.596023    2583 generic.go:356] "Generic (PLEG): container finished" podID="a1894a02-4494-4cc0-970d-ba97f2309187" containerID="396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b" exitCode=0
Sep 11 19:39:24.596293 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.596072    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"
Sep 11 19:39:24.596293 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.596079    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt" event={"ID":"a1894a02-4494-4cc0-970d-ba97f2309187","Type":"ContainerDied","Data":"396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b"}
Sep 11 19:39:24.596293 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.596132    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt" event={"ID":"a1894a02-4494-4cc0-970d-ba97f2309187","Type":"ContainerDied","Data":"e72dd2f5a5896ce535b708f51323a3c47cf60ed21ec13aeb4cb84f100f9f96ad"}
Sep 11 19:39:24.596293 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.596159    2583 scope.go:117] "RemoveContainer" containerID="396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b"
Sep 11 19:39:24.605190 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.605165    2583 scope.go:117] "RemoveContainer" containerID="396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b"
Sep 11 19:39:24.605618 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:39:24.605597    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b\": container with ID starting with 396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b not found: ID does not exist" containerID="396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b"
Sep 11 19:39:24.605722 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.605645    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b"} err="failed to get container status \"396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b\": rpc error: code = NotFound desc = could not find container \"396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b\": container with ID starting with 396e7ffc09f33f10ee6a696a4d3537cf7fa4fd7d09df78fb492941bd5ee88f5b not found: ID does not exist"
Sep 11 19:39:24.647414 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.647385    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1894a02-4494-4cc0-970d-ba97f2309187-plugins-config-volume\") pod \"a1894a02-4494-4cc0-970d-ba97f2309187\" (UID: \"a1894a02-4494-4cc0-970d-ba97f2309187\") "
Sep 11 19:39:24.647570 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.647470    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf66b\" (UniqueName: \"kubernetes.io/projected/a1894a02-4494-4cc0-970d-ba97f2309187-kube-api-access-mf66b\") pod \"a1894a02-4494-4cc0-970d-ba97f2309187\" (UID: \"a1894a02-4494-4cc0-970d-ba97f2309187\") "
Sep 11 19:39:24.647759 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.647731    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1894a02-4494-4cc0-970d-ba97f2309187-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1894a02-4494-4cc0-970d-ba97f2309187" (UID: "a1894a02-4494-4cc0-970d-ba97f2309187"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:39:24.649738 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.649708    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1894a02-4494-4cc0-970d-ba97f2309187-kube-api-access-mf66b" (OuterVolumeSpecName: "kube-api-access-mf66b") pod "a1894a02-4494-4cc0-970d-ba97f2309187" (UID: "a1894a02-4494-4cc0-970d-ba97f2309187"). InnerVolumeSpecName "kube-api-access-mf66b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:39:24.748532 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.748429    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf66b\" (UniqueName: \"kubernetes.io/projected/a1894a02-4494-4cc0-970d-ba97f2309187-kube-api-access-mf66b\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:24.748532 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.748477    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1894a02-4494-4cc0-970d-ba97f2309187-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:24.770566 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.770527    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:24.872912 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.872873    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:24.925325 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.925299    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"]
Sep 11 19:39:24.932705 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:24.932675    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-515a4f-6cc9899f6c-vv9gt"]
Sep 11 19:39:25.647592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:25.647562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:26.378116 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:26.378076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:26.546292 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:26.546256    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1894a02-4494-4cc0-970d-ba97f2309187" path="/var/lib/kubelet/pods/a1894a02-4494-4cc0-970d-ba97f2309187/volumes"
Sep 11 19:39:27.129742 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:27.129703    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:27.879458 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:27.879404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:30.600300 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:30.600235    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:30.701756 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:30.701719    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:30.860100 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:30.860022    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:31.371520 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:31.371477    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:31.682186 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:31.682102    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:32.186650 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:32.186609    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:34.035126 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:34.035091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:34.709140 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:34.709102    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:34.789991 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:34.789949    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:35.003795 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:35.003712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:35.735483 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:35.735450    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:36.643417 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:36.643382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:37.423123 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:37.423096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:38.155759 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.155730    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:38.255806 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.255772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:38.356369 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.356334    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:38.612505 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.612471    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:38.709329 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.709298    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:38.779525 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.779485    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"]
Sep 11 19:39:38.779920 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.779907    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1894a02-4494-4cc0-970d-ba97f2309187" containerName="payload-processing"
Sep 11 19:39:38.779964 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.779924    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1894a02-4494-4cc0-970d-ba97f2309187" containerName="payload-processing"
Sep 11 19:39:38.779964 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.779938    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93f63ac-6153-4c47-acce-61cb35522d73" containerName="main"
Sep 11 19:39:38.779964 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.779944    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93f63ac-6153-4c47-acce-61cb35522d73" containerName="main"
Sep 11 19:39:38.779964 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.779957    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf" containerName="payload-processing"
Sep 11 19:39:38.779964 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.779965    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf" containerName="payload-processing"
Sep 11 19:39:38.780117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.779975    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77daedc1-6d88-4c54-a00a-d6039d9474c9" containerName="maas-api"
Sep 11 19:39:38.780117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.779980    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="77daedc1-6d88-4c54-a00a-d6039d9474c9" containerName="maas-api"
Sep 11 19:39:38.780117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.779989    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d14a765-b470-4422-88d0-a7dd292f277f" containerName="payload-pre-processing"
Sep 11 19:39:38.780117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.779994    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d14a765-b470-4422-88d0-a7dd292f277f" containerName="payload-pre-processing"
Sep 11 19:39:38.780117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.780069    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1894a02-4494-4cc0-970d-ba97f2309187" containerName="payload-processing"
Sep 11 19:39:38.780117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.780077    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d14a765-b470-4422-88d0-a7dd292f277f" containerName="payload-pre-processing"
Sep 11 19:39:38.780117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.780088    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="32e5a0bd-966f-4fcb-b5b9-fe630cf0edaf" containerName="payload-processing"
Sep 11 19:39:38.780117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.780100    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="77daedc1-6d88-4c54-a00a-d6039d9474c9" containerName="maas-api"
Sep 11 19:39:38.780117 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.780106    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93f63ac-6153-4c47-acce-61cb35522d73" containerName="main"
Sep 11 19:39:38.785383 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.785229    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:39:38.789440 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.789220    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-19859a\""
Sep 11 19:39:38.789734 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.789707    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-19859a-dockercfg-4ncfh\""
Sep 11 19:39:38.794028 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.794002    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"]
Sep 11 19:39:38.888826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.888718    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1af2b69e-5830-40bf-87e6-81efaded35ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs\" (UID: \"1af2b69e-5830-40bf-87e6-81efaded35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:39:38.888826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.888769    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df55p\" (UniqueName: \"kubernetes.io/projected/1af2b69e-5830-40bf-87e6-81efaded35ad-kube-api-access-df55p\") pod \"payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs\" (UID: \"1af2b69e-5830-40bf-87e6-81efaded35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:39:38.989669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.989634    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1af2b69e-5830-40bf-87e6-81efaded35ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs\" (UID: \"1af2b69e-5830-40bf-87e6-81efaded35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:39:38.989845 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.989677    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df55p\" (UniqueName: \"kubernetes.io/projected/1af2b69e-5830-40bf-87e6-81efaded35ad-kube-api-access-df55p\") pod \"payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs\" (UID: \"1af2b69e-5830-40bf-87e6-81efaded35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:39:38.990377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.990358    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1af2b69e-5830-40bf-87e6-81efaded35ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs\" (UID: \"1af2b69e-5830-40bf-87e6-81efaded35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:39:38.997686 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:38.997664    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df55p\" (UniqueName: \"kubernetes.io/projected/1af2b69e-5830-40bf-87e6-81efaded35ad-kube-api-access-df55p\") pod \"payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs\" (UID: \"1af2b69e-5830-40bf-87e6-81efaded35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:39:39.099672 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:39.099641    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:39:39.371852 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:39.371825    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"]
Sep 11 19:39:39.373876 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:39:39.373849    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1af2b69e_5830_40bf_87e6_81efaded35ad.slice/crio-5f8928c7a156cd79b5dc1c711b512372e866963142b20c2a782e5918a9d47f1d WatchSource:0}: Error finding container 5f8928c7a156cd79b5dc1c711b512372e866963142b20c2a782e5918a9d47f1d: Status 404 returned error can't find the container with id 5f8928c7a156cd79b5dc1c711b512372e866963142b20c2a782e5918a9d47f1d
Sep 11 19:39:39.675745 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:39.675651    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs" event={"ID":"1af2b69e-5830-40bf-87e6-81efaded35ad","Type":"ContainerStarted","Data":"f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5"}
Sep 11 19:39:39.675745 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:39.675715    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs" event={"ID":"1af2b69e-5830-40bf-87e6-81efaded35ad","Type":"ContainerStarted","Data":"5f8928c7a156cd79b5dc1c711b512372e866963142b20c2a782e5918a9d47f1d"}
Sep 11 19:39:39.676165 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:39.675827    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:39:39.693467 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:39.693415    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs" podStartSLOduration=1.693399754 podStartE2EDuration="1.693399754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:39:39.692016877 +0000 UTC m=+1283.620641776" watchObservedRunningTime="2026-09-11 19:39:39.693399754 +0000 UTC m=+1283.622024698"
Sep 11 19:39:40.253538 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:40.253494    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:41.132184 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:41.132154    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:42.021663 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:42.021625    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:42.667446 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:42.667416    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:43.332867 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:43.332835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:44.256158 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:44.256127    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:44.928875 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:44.928840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:50.683351 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:50.683317    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:39:55.672082 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:55.672047    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"]
Sep 11 19:39:55.672499 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:55.672287    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws" podUID="0770c927-94aa-4ff7-b1e3-b44e28cdcd8c" containerName="maas-api" containerID="cri-o://7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e" gracePeriod=30
Sep 11 19:39:55.704926 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:55.704869    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:55.729036 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:55.728999    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"]
Sep 11 19:39:55.729235 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:55.729213    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67" podUID="c879b361-9e26-4a3d-ac39-33dc7ea1e800" containerName="payload-processing" containerID="cri-o://3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b" gracePeriod=30
Sep 11 19:39:55.738259 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:55.738212    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"]
Sep 11 19:39:55.738629 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:55.738593    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh" podUID="b2195476-40b4-4785-b27b-8fe3ece5f799" containerName="payload-pre-processing" containerID="cri-o://000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2" gracePeriod=30
Sep 11 19:39:55.770701 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:55.770670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:55.827777 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:55.827736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:55.909640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:55.909538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:55.966412 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:55.966345    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:56.043144 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.043120    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:39:56.046299 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.046278    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:39:56.049590 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.049574    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:39:56.154592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.154564    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgcc2\" (UniqueName: \"kubernetes.io/projected/c879b361-9e26-4a3d-ac39-33dc7ea1e800-kube-api-access-hgcc2\") pod \"c879b361-9e26-4a3d-ac39-33dc7ea1e800\" (UID: \"c879b361-9e26-4a3d-ac39-33dc7ea1e800\") "
Sep 11 19:39:56.154592 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.154597    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-openshift-service-ca\") pod \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") "
Sep 11 19:39:56.154846 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.154627    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds9vj\" (UniqueName: \"kubernetes.io/projected/b2195476-40b4-4785-b27b-8fe3ece5f799-kube-api-access-ds9vj\") pod \"b2195476-40b4-4785-b27b-8fe3ece5f799\" (UID: \"b2195476-40b4-4785-b27b-8fe3ece5f799\") "
Sep 11 19:39:56.154846 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.154648    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-metrics-certs\") pod \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") "
Sep 11 19:39:56.154846 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.154734    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc77t\" (UniqueName: \"kubernetes.io/projected/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-kube-api-access-cc77t\") pod \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") "
Sep 11 19:39:56.154846 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.154766    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c879b361-9e26-4a3d-ac39-33dc7ea1e800-plugins-config-volume\") pod \"c879b361-9e26-4a3d-ac39-33dc7ea1e800\" (UID: \"c879b361-9e26-4a3d-ac39-33dc7ea1e800\") "
Sep 11 19:39:56.154846 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.154833    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-maas-api-tls\") pod \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\" (UID: \"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c\") "
Sep 11 19:39:56.155101 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.154863    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2195476-40b4-4785-b27b-8fe3ece5f799-plugins-config-volume\") pod \"b2195476-40b4-4785-b27b-8fe3ece5f799\" (UID: \"b2195476-40b4-4785-b27b-8fe3ece5f799\") "
Sep 11 19:39:56.155101 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.154978    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0770c927-94aa-4ff7-b1e3-b44e28cdcd8c" (UID: "0770c927-94aa-4ff7-b1e3-b44e28cdcd8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:39:56.155218 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.155191    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c879b361-9e26-4a3d-ac39-33dc7ea1e800-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c879b361-9e26-4a3d-ac39-33dc7ea1e800" (UID: "c879b361-9e26-4a3d-ac39-33dc7ea1e800"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:39:56.155341 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.155323    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:56.155399 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.155347    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c879b361-9e26-4a3d-ac39-33dc7ea1e800-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:56.155822 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.155789    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2195476-40b4-4785-b27b-8fe3ece5f799-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2195476-40b4-4785-b27b-8fe3ece5f799" (UID: "b2195476-40b4-4785-b27b-8fe3ece5f799"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:39:56.157147 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.157118    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0770c927-94aa-4ff7-b1e3-b44e28cdcd8c" (UID: "0770c927-94aa-4ff7-b1e3-b44e28cdcd8c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:39:56.157827 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.157807    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0770c927-94aa-4ff7-b1e3-b44e28cdcd8c" (UID: "0770c927-94aa-4ff7-b1e3-b44e28cdcd8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:39:56.157921 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.157893    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2195476-40b4-4785-b27b-8fe3ece5f799-kube-api-access-ds9vj" (OuterVolumeSpecName: "kube-api-access-ds9vj") pod "b2195476-40b4-4785-b27b-8fe3ece5f799" (UID: "b2195476-40b4-4785-b27b-8fe3ece5f799"). InnerVolumeSpecName "kube-api-access-ds9vj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:39:56.157983 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.157902    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-kube-api-access-cc77t" (OuterVolumeSpecName: "kube-api-access-cc77t") pod "0770c927-94aa-4ff7-b1e3-b44e28cdcd8c" (UID: "0770c927-94aa-4ff7-b1e3-b44e28cdcd8c"). InnerVolumeSpecName "kube-api-access-cc77t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:39:56.157983 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.157937    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c879b361-9e26-4a3d-ac39-33dc7ea1e800-kube-api-access-hgcc2" (OuterVolumeSpecName: "kube-api-access-hgcc2") pod "c879b361-9e26-4a3d-ac39-33dc7ea1e800" (UID: "c879b361-9e26-4a3d-ac39-33dc7ea1e800"). InnerVolumeSpecName "kube-api-access-hgcc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:39:56.255910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.255821    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgcc2\" (UniqueName: \"kubernetes.io/projected/c879b361-9e26-4a3d-ac39-33dc7ea1e800-kube-api-access-hgcc2\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:56.255910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.255852    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds9vj\" (UniqueName: \"kubernetes.io/projected/b2195476-40b4-4785-b27b-8fe3ece5f799-kube-api-access-ds9vj\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:56.255910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.255874    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:56.255910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.255882    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc77t\" (UniqueName: \"kubernetes.io/projected/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-kube-api-access-cc77t\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:56.255910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.255892    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:56.255910 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.255905    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2195476-40b4-4785-b27b-8fe3ece5f799-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:39:56.327533 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.327498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:56.764689 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.764654    2583 generic.go:356] "Generic (PLEG): container finished" podID="0770c927-94aa-4ff7-b1e3-b44e28cdcd8c" containerID="7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e" exitCode=0
Sep 11 19:39:56.765104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.764715    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"
Sep 11 19:39:56.765104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.764739    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws" event={"ID":"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c","Type":"ContainerDied","Data":"7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e"}
Sep 11 19:39:56.765104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.764795    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws" event={"ID":"0770c927-94aa-4ff7-b1e3-b44e28cdcd8c","Type":"ContainerDied","Data":"868a0ce9f5ebd12029e758de7718ced458a7ce2aecc277b8d6699acc36e97599"}
Sep 11 19:39:56.765104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.764818    2583 scope.go:117] "RemoveContainer" containerID="7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e"
Sep 11 19:39:56.766229 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.766181    2583 generic.go:356] "Generic (PLEG): container finished" podID="b2195476-40b4-4785-b27b-8fe3ece5f799" containerID="000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2" exitCode=0
Sep 11 19:39:56.766229 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.766215    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh" event={"ID":"b2195476-40b4-4785-b27b-8fe3ece5f799","Type":"ContainerDied","Data":"000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2"}
Sep 11 19:39:56.766401 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.766268    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"
Sep 11 19:39:56.766401 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.766279    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh" event={"ID":"b2195476-40b4-4785-b27b-8fe3ece5f799","Type":"ContainerDied","Data":"7f20a0d48de20bcc4746f04ef231fe9397b380b5da00e811b700ba42dd614e73"}
Sep 11 19:39:56.767852 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.767812    2583 generic.go:356] "Generic (PLEG): container finished" podID="c879b361-9e26-4a3d-ac39-33dc7ea1e800" containerID="3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b" exitCode=0
Sep 11 19:39:56.767993 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.767862    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67" event={"ID":"c879b361-9e26-4a3d-ac39-33dc7ea1e800","Type":"ContainerDied","Data":"3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b"}
Sep 11 19:39:56.767993 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.767883    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"
Sep 11 19:39:56.767993 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.767946    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67" event={"ID":"c879b361-9e26-4a3d-ac39-33dc7ea1e800","Type":"ContainerDied","Data":"2467070b49167458933d9cc32a33d66287e496b429fa8610fed41c4032001c7a"}
Sep 11 19:39:56.774981 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.774964    2583 scope.go:117] "RemoveContainer" containerID="7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e"
Sep 11 19:39:56.775230 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:39:56.775211    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e\": container with ID starting with 7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e not found: ID does not exist" containerID="7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e"
Sep 11 19:39:56.775298 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.775253    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e"} err="failed to get container status \"7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e\": rpc error: code = NotFound desc = could not find container \"7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e\": container with ID starting with 7d363b7e973964dafe868d30a696c4a104a8e0cd86b05a9b51d8a6c5198d507e not found: ID does not exist"
Sep 11 19:39:56.775298 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.775268    2583 scope.go:117] "RemoveContainer" containerID="000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2"
Sep 11 19:39:56.783392 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.783376    2583 scope.go:117] "RemoveContainer" containerID="000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2"
Sep 11 19:39:56.783630 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:39:56.783614    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2\": container with ID starting with 000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2 not found: ID does not exist" containerID="000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2"
Sep 11 19:39:56.783669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.783637    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2"} err="failed to get container status \"000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2\": rpc error: code = NotFound desc = could not find container \"000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2\": container with ID starting with 000302e2602f158a456c343506595aeebd3d922eaaaa9cc176ef187cbd0e97e2 not found: ID does not exist"
Sep 11 19:39:56.783669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.783652    2583 scope.go:117] "RemoveContainer" containerID="3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b"
Sep 11 19:39:56.791881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.791857    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"]
Sep 11 19:39:56.792997 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.792980    2583 scope.go:117] "RemoveContainer" containerID="3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b"
Sep 11 19:39:56.793323 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:39:56.793303    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b\": container with ID starting with 3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b not found: ID does not exist" containerID="3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b"
Sep 11 19:39:56.793416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.793328    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b"} err="failed to get container status \"3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b\": rpc error: code = NotFound desc = could not find container \"3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b\": container with ID starting with 3f1922085a11e0ab57ba35959ac04c505f9bed191496b5a036c29037053eaf6b not found: ID does not exist"
Sep 11 19:39:56.794230 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.794209    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-55586842-84485f948f-xnx67"]
Sep 11 19:39:56.803903 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.803880    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"]
Sep 11 19:39:56.810448 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.810400    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-55586842-6bb6f4566b-gt4qh"]
Sep 11 19:39:56.819407 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.819388    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"]
Sep 11 19:39:56.821869 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:56.821844    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-55586842-f9cd5d6dd-j9xws"]
Sep 11 19:39:57.049114 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:57.049069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:57.825988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:57.825959    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:58.547087 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:58.547052    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0770c927-94aa-4ff7-b1e3-b44e28cdcd8c" path="/var/lib/kubelet/pods/0770c927-94aa-4ff7-b1e3-b44e28cdcd8c/volumes"
Sep 11 19:39:58.547484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:58.547469    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2195476-40b4-4785-b27b-8fe3ece5f799" path="/var/lib/kubelet/pods/b2195476-40b4-4785-b27b-8fe3ece5f799/volumes"
Sep 11 19:39:58.547786 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:58.547774    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c879b361-9e26-4a3d-ac39-33dc7ea1e800" path="/var/lib/kubelet/pods/c879b361-9e26-4a3d-ac39-33dc7ea1e800/volumes"
Sep 11 19:39:58.745456 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:58.745421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:58.989293 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:58.989235    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:39:59.607906 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:39:59.607868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:00.426012 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:00.425972    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:01.100821 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:01.100781    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:04.048805 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:04.048763    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:09.543966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:09.543928    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:09.849867 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:09.849839    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:10.164685 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:10.164613    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:10.465755 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:10.465674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:12.108775 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:12.108746    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:12.821373 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:12.821340    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:12.915533 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:12.915496    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:13.015499 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:13.015461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:13.755393 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:13.755364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:14.524719 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:14.524686    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:15.350668 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:15.350630    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:16.008986 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.008954    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"]
Sep 11 19:40:16.009544 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.009520    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c879b361-9e26-4a3d-ac39-33dc7ea1e800" containerName="payload-processing"
Sep 11 19:40:16.009712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.009604    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c879b361-9e26-4a3d-ac39-33dc7ea1e800" containerName="payload-processing"
Sep 11 19:40:16.009712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.009667    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0770c927-94aa-4ff7-b1e3-b44e28cdcd8c" containerName="maas-api"
Sep 11 19:40:16.009712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.009677    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0770c927-94aa-4ff7-b1e3-b44e28cdcd8c" containerName="maas-api"
Sep 11 19:40:16.009712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.009691    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2195476-40b4-4785-b27b-8fe3ece5f799" containerName="payload-pre-processing"
Sep 11 19:40:16.009712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.009699    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2195476-40b4-4785-b27b-8fe3ece5f799" containerName="payload-pre-processing"
Sep 11 19:40:16.009984 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.009849    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0770c927-94aa-4ff7-b1e3-b44e28cdcd8c" containerName="maas-api"
Sep 11 19:40:16.009984 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.009866    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2195476-40b4-4785-b27b-8fe3ece5f799" containerName="payload-pre-processing"
Sep 11 19:40:16.009984 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.009893    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c879b361-9e26-4a3d-ac39-33dc7ea1e800" containerName="payload-processing"
Sep 11 19:40:16.013317 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.013296    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.015576 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.015552    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-abcdff-serving-cert\""
Sep 11 19:40:16.022952 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.022925    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"]
Sep 11 19:40:16.040836 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.040812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:16.041004 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.040897    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/283f6644-486d-4878-9545-8536d99f4ad5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.041075 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.040998    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.041138 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.041073    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.041138 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.041115    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnl7r\" (UniqueName: \"kubernetes.io/projected/283f6644-486d-4878-9545-8536d99f4ad5-kube-api-access-bnl7r\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.103604 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.103574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:16.142384 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.142353    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.142574 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.142407    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.142655 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.142579    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnl7r\" (UniqueName: \"kubernetes.io/projected/283f6644-486d-4878-9545-8536d99f4ad5-kube-api-access-bnl7r\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.142655 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.142647    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/283f6644-486d-4878-9545-8536d99f4ad5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.143231 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.143207    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/283f6644-486d-4878-9545-8536d99f4ad5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.145065 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.145044    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.145170 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.145051    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.150290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.150268    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnl7r\" (UniqueName: \"kubernetes.io/projected/283f6644-486d-4878-9545-8536d99f4ad5-kube-api-access-bnl7r\") pod \"maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.204018 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.203983    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:16.327706 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.327607    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:16.496119 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.496086    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:16.591284 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.591190    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:16.600911 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.600890    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"]
Sep 11 19:40:16.603732 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:40:16.603699    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod283f6644_486d_4878_9545_8536d99f4ad5.slice/crio-1aada969e8bfb8d9455d2d82f32e4cb157581d79fcdbeaeeafed525fa32eedda WatchSource:0}: Error finding container 1aada969e8bfb8d9455d2d82f32e4cb157581d79fcdbeaeeafed525fa32eedda: Status 404 returned error can't find the container with id 1aada969e8bfb8d9455d2d82f32e4cb157581d79fcdbeaeeafed525fa32eedda
Sep 11 19:40:16.605502 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.605479    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"]
Sep 11 19:40:16.610291 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.610270    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:40:16.612626 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.612603    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-abcdff\""
Sep 11 19:40:16.612788 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.612770    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-abcdff-dockercfg-rmjvk\""
Sep 11 19:40:16.621821 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.621600    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"]
Sep 11 19:40:16.631421 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.631390    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"]
Sep 11 19:40:16.634881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.634859    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:40:16.646651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.646630    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"]
Sep 11 19:40:16.646896 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.646876    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6668\" (UniqueName: \"kubernetes.io/projected/8e1024cb-256f-4317-8f29-a0ef4f0b491a-kube-api-access-v6668\") pod \"payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws\" (UID: \"8e1024cb-256f-4317-8f29-a0ef4f0b491a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:40:16.646953 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.646907    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1024cb-256f-4317-8f29-a0ef4f0b491a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws\" (UID: \"8e1024cb-256f-4317-8f29-a0ef4f0b491a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:40:16.748085 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.748055    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06869ae7-9c43-4b03-9b32-490b86d7dba9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz\" (UID: \"06869ae7-9c43-4b03-9b32-490b86d7dba9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:40:16.748282 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.748113    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-655ts\" (UniqueName: \"kubernetes.io/projected/06869ae7-9c43-4b03-9b32-490b86d7dba9-kube-api-access-655ts\") pod \"payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz\" (UID: \"06869ae7-9c43-4b03-9b32-490b86d7dba9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:40:16.748282 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.748196    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6668\" (UniqueName: \"kubernetes.io/projected/8e1024cb-256f-4317-8f29-a0ef4f0b491a-kube-api-access-v6668\") pod \"payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws\" (UID: \"8e1024cb-256f-4317-8f29-a0ef4f0b491a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:40:16.748282 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.748216    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1024cb-256f-4317-8f29-a0ef4f0b491a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws\" (UID: \"8e1024cb-256f-4317-8f29-a0ef4f0b491a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:40:16.748897 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.748879    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1024cb-256f-4317-8f29-a0ef4f0b491a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws\" (UID: \"8e1024cb-256f-4317-8f29-a0ef4f0b491a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:40:16.756070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.756042    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6668\" (UniqueName: \"kubernetes.io/projected/8e1024cb-256f-4317-8f29-a0ef4f0b491a-kube-api-access-v6668\") pod \"payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws\" (UID: \"8e1024cb-256f-4317-8f29-a0ef4f0b491a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:40:16.848854 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.848813    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06869ae7-9c43-4b03-9b32-490b86d7dba9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz\" (UID: \"06869ae7-9c43-4b03-9b32-490b86d7dba9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:40:16.849055 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.848891    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-655ts\" (UniqueName: \"kubernetes.io/projected/06869ae7-9c43-4b03-9b32-490b86d7dba9-kube-api-access-655ts\") pod \"payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz\" (UID: \"06869ae7-9c43-4b03-9b32-490b86d7dba9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:40:16.849808 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.849780    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06869ae7-9c43-4b03-9b32-490b86d7dba9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz\" (UID: \"06869ae7-9c43-4b03-9b32-490b86d7dba9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:40:16.857481 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.857452    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-655ts\" (UniqueName: \"kubernetes.io/projected/06869ae7-9c43-4b03-9b32-490b86d7dba9-kube-api-access-655ts\") pod \"payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz\" (UID: \"06869ae7-9c43-4b03-9b32-490b86d7dba9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:40:16.872734 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.872699    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s" event={"ID":"283f6644-486d-4878-9545-8536d99f4ad5","Type":"ContainerStarted","Data":"1aada969e8bfb8d9455d2d82f32e4cb157581d79fcdbeaeeafed525fa32eedda"}
Sep 11 19:40:16.925779 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.925743    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:40:16.945279 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:16.945230    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:40:17.202460 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.202416    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"]
Sep 11 19:40:17.205383 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:40:17.205357    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e1024cb_256f_4317_8f29_a0ef4f0b491a.slice/crio-0ba6cf19e948984c21e7a59e77b6bb08eedf17227d81428da3f51ad2a7dde4e9 WatchSource:0}: Error finding container 0ba6cf19e948984c21e7a59e77b6bb08eedf17227d81428da3f51ad2a7dde4e9: Status 404 returned error can't find the container with id 0ba6cf19e948984c21e7a59e77b6bb08eedf17227d81428da3f51ad2a7dde4e9
Sep 11 19:40:17.232465 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.232387    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"]
Sep 11 19:40:17.234708 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:40:17.234683    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06869ae7_9c43_4b03_9b32_490b86d7dba9.slice/crio-97062426fab0561b440ce834c84889f80269c5f99a5d1569cc56360945133019 WatchSource:0}: Error finding container 97062426fab0561b440ce834c84889f80269c5f99a5d1569cc56360945133019: Status 404 returned error can't find the container with id 97062426fab0561b440ce834c84889f80269c5f99a5d1569cc56360945133019
Sep 11 19:40:17.878974 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.878941    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws" event={"ID":"8e1024cb-256f-4317-8f29-a0ef4f0b491a","Type":"ContainerStarted","Data":"f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0"}
Sep 11 19:40:17.879463 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.879040    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws" event={"ID":"8e1024cb-256f-4317-8f29-a0ef4f0b491a","Type":"ContainerStarted","Data":"0ba6cf19e948984c21e7a59e77b6bb08eedf17227d81428da3f51ad2a7dde4e9"}
Sep 11 19:40:17.879463 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.879073    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:40:17.880612 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.880579    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s" event={"ID":"283f6644-486d-4878-9545-8536d99f4ad5","Type":"ContainerStarted","Data":"fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587"}
Sep 11 19:40:17.880751 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.880682    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:17.882255 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.882214    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz" event={"ID":"06869ae7-9c43-4b03-9b32-490b86d7dba9","Type":"ContainerStarted","Data":"8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a"}
Sep 11 19:40:17.882376 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.882283    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz" event={"ID":"06869ae7-9c43-4b03-9b32-490b86d7dba9","Type":"ContainerStarted","Data":"97062426fab0561b440ce834c84889f80269c5f99a5d1569cc56360945133019"}
Sep 11 19:40:17.882376 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.882330    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:40:17.894664 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.894623    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws" podStartSLOduration=1.894608872 podStartE2EDuration="1.894608872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:40:17.89360886 +0000 UTC m=+1321.822233759" watchObservedRunningTime="2026-09-11 19:40:17.894608872 +0000 UTC m=+1321.823233770"
Sep 11 19:40:17.910349 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.910305    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s" podStartSLOduration=2.47203996 podStartE2EDuration="2.910289856s" totalImagesPullingTime="438.249896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:40:17.908995864 +0000 UTC m=+1321.837620761" watchObservedRunningTime="2026-09-11 19:40:17.910289856 +0000 UTC m=+1321.838914767"
Sep 11 19:40:17.924934 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:17.924887    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz" podStartSLOduration=1.924872636 podStartE2EDuration="1.924872636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:40:17.923988145 +0000 UTC m=+1321.852613044" watchObservedRunningTime="2026-09-11 19:40:17.924872636 +0000 UTC m=+1321.853497538"
Sep 11 19:40:23.894015 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:23.893981    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:40:28.890582 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:28.890549    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:40:28.891060 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:28.890625    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:40:37.805951 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:37.805846    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:37.962591 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:37.962558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:38.837419 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:38.837385    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:39.502022 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:39.501986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:40.362459 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:40.362429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:40.462418 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:40.462387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:41.158147 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:41.158118    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:41.693351 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:41.693312    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:41.852540 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:41.852498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:47.531051 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:47.531015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:48.999640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:48.999606    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"]
Sep 11 19:40:49.003407 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.003387    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.006892 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.006866    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-19859a\"/\"kube-root-ca.crt\""
Sep 11 19:40:49.007040 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.006902    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-19859a\"/\"openshift-service-ca.crt\""
Sep 11 19:40:49.007040 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.006977    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-19859a\"/\"default-dockercfg-l7jwf\""
Sep 11 19:40:49.007262 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.007222    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-19859a\"/\"auth-test-model-19859a-kserve-self-signed-certs\""
Sep 11 19:40:49.012807 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.012783    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"]
Sep 11 19:40:49.028009 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.027979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:49.056677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.056650    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpqml\" (UniqueName: \"kubernetes.io/projected/c83f28ea-8770-4fd1-9708-9dea39965fe3-kube-api-access-qpqml\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.056829 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.056692    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c83f28ea-8770-4fd1-9708-9dea39965fe3-tls-certs\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.056829 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.056778    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-model-cache\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.056829 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.056814    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-dshm\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.056966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.056845    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-tmp-dir\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.056966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.056875    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-home\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.157714 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.157672    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-model-cache\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.157914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.157727    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-dshm\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.157914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.157764    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-tmp-dir\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.157914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.157792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-home\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.157914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.157862    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpqml\" (UniqueName: \"kubernetes.io/projected/c83f28ea-8770-4fd1-9708-9dea39965fe3-kube-api-access-qpqml\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.157914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.157891    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c83f28ea-8770-4fd1-9708-9dea39965fe3-tls-certs\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.158184 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.158144    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-model-cache\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.158285 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.158179    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-tmp-dir\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.158352 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.158280    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-home\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.160281 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.160235    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-dshm\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.160462 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.160447    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c83f28ea-8770-4fd1-9708-9dea39965fe3-tls-certs\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.165539 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.165519    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpqml\" (UniqueName: \"kubernetes.io/projected/c83f28ea-8770-4fd1-9708-9dea39965fe3-kube-api-access-qpqml\") pod \"auth-test-model-19859a-kserve-f6bd947c9-j2cm8\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") " pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.315211 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.314867    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:49.589990 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:49.589967    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"]
Sep 11 19:40:49.592172 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:40:49.592133    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc83f28ea_8770_4fd1_9708_9dea39965fe3.slice/crio-c326c7b3125a23b539319a2c1db0374eb36948235266277e1c339b76e630899c WatchSource:0}: Error finding container c326c7b3125a23b539319a2c1db0374eb36948235266277e1c339b76e630899c: Status 404 returned error can't find the container with id c326c7b3125a23b539319a2c1db0374eb36948235266277e1c339b76e630899c
Sep 11 19:40:50.039614 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:50.039575    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8" event={"ID":"c83f28ea-8770-4fd1-9708-9dea39965fe3","Type":"ContainerStarted","Data":"446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5"}
Sep 11 19:40:50.039997 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:50.039625    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8" event={"ID":"c83f28ea-8770-4fd1-9708-9dea39965fe3","Type":"ContainerStarted","Data":"c326c7b3125a23b539319a2c1db0374eb36948235266277e1c339b76e630899c"}
Sep 11 19:40:50.056146 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:50.056102    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8" podStartSLOduration=2.056086799 podStartE2EDuration="2.056086799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:40:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:40:50.053998869 +0000 UTC m=+1353.982623767" watchObservedRunningTime="2026-09-11 19:40:50.056086799 +0000 UTC m=+1353.984711697"
Sep 11 19:40:53.073374 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:53.073346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:53.131980 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:53.131934    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:53.337042 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:53.336961    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:54.632947 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:54.632877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:54.732362 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:54.732331    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:54.832565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:54.832528    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:55.466175 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:55.466144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:55.564215 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:55.564182    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:55.666335 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:55.666294    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:56.955488 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:56.955445    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:40:59.315896 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:59.315860    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:59.316398 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:59.315919    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:40:59.328813 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:40:59.328789    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:41:00.101372 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:00.101341    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:41:02.643908 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:02.643870    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:02.707569 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:02.707537    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:03.309842 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:03.309810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:08.898415 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:08.898381    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:09.006706 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:09.006674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:09.098878 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:09.098847    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:16.932098 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:16.932064    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:17.056386 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:17.056348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:19.646996 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:19.646969    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:19.880971 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:19.880939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:21.723444 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:21.723404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:24.618861 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.618829    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"]
Sep 11 19:41:24.622735 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.622719    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.624936 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.624911    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1fc808\"/\"embedding-w0-1fc808-kserve-self-signed-certs\""
Sep 11 19:41:24.629867 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.629844    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"]
Sep 11 19:41:24.708225 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.708192    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-model-cache\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.708451 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.708262    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-tmp-dir\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.708451 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.708360    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb451d72-d2c8-4796-979e-4a00420ab008-tls-certs\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.708595 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.708520    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-home\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.708595 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.708554    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-dshm\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.708595 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.708580    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j2h8\" (UniqueName: \"kubernetes.io/projected/bb451d72-d2c8-4796-979e-4a00420ab008-kube-api-access-4j2h8\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.709971 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.709940    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:24.810173 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.810133    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-model-cache\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.810377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.810189    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-tmp-dir\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.810377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.810223    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb451d72-d2c8-4796-979e-4a00420ab008-tls-certs\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.810377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.810289    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-home\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.810377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.810310    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-dshm\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.810377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.810326    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j2h8\" (UniqueName: \"kubernetes.io/projected/bb451d72-d2c8-4796-979e-4a00420ab008-kube-api-access-4j2h8\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.810674 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.810651    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-model-cache\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.810730 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.810676    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-tmp-dir\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.810954 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.810928    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-home\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.813360 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.813336    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-dshm\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.813618 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.813601    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb451d72-d2c8-4796-979e-4a00420ab008-tls-certs\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.818080 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.818057    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j2h8\" (UniqueName: \"kubernetes.io/projected/bb451d72-d2c8-4796-979e-4a00420ab008-kube-api-access-4j2h8\") pod \"embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") " pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:24.932860 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:24.932768    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:25.207411 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:25.207386    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"]
Sep 11 19:41:25.209887 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:41:25.209851    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb451d72_d2c8_4796_979e_4a00420ab008.slice/crio-33435ad132ac6d63e7a1dfa1ac4b73c2eff6d58de4182fe553fbc8f056af97cc WatchSource:0}: Error finding container 33435ad132ac6d63e7a1dfa1ac4b73c2eff6d58de4182fe553fbc8f056af97cc: Status 404 returned error can't find the container with id 33435ad132ac6d63e7a1dfa1ac4b73c2eff6d58de4182fe553fbc8f056af97cc
Sep 11 19:41:26.221458 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:26.221424    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q" event={"ID":"bb451d72-d2c8-4796-979e-4a00420ab008","Type":"ContainerStarted","Data":"238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa"}
Sep 11 19:41:26.221855 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:26.221473    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q" event={"ID":"bb451d72-d2c8-4796-979e-4a00420ab008","Type":"ContainerStarted","Data":"33435ad132ac6d63e7a1dfa1ac4b73c2eff6d58de4182fe553fbc8f056af97cc"}
Sep 11 19:41:26.238518 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:26.238470    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q" podStartSLOduration=2.238453648 podStartE2EDuration="2.238453648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:41:26.236913033 +0000 UTC m=+1390.165537932" watchObservedRunningTime="2026-09-11 19:41:26.238453648 +0000 UTC m=+1390.167078545"
Sep 11 19:41:32.616053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.616014    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"]
Sep 11 19:41:32.619948 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.619928    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.623389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.623364    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-abcdff\"/\"openshift-service-ca.crt\""
Sep 11 19:41:32.623573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.623365    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-abcdff\"/\"default-dockercfg-lv6pr\""
Sep 11 19:41:32.623573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.623428    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-abcdff\"/\"auth-test-model-abcdff-kserve-self-signed-certs\""
Sep 11 19:41:32.623573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.623438    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-abcdff\"/\"kube-root-ca.crt\""
Sep 11 19:41:32.626778 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.626755    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"]
Sep 11 19:41:32.632765 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.632743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:32.684433 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.684396    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-home\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.684618 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.684439    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5h6k\" (UniqueName: \"kubernetes.io/projected/781a9e72-e05b-4bb1-9a3d-a6b960d37809-kube-api-access-l5h6k\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.684618 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.684470    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-model-cache\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.684618 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.684553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tmp-dir\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.684618 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.684585    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tls-certs\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.684763 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.684686    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-dshm\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.785706 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.785665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-dshm\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.785894 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.785717    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-home\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.785894 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.785738    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5h6k\" (UniqueName: \"kubernetes.io/projected/781a9e72-e05b-4bb1-9a3d-a6b960d37809-kube-api-access-l5h6k\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.786009 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.785944    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-model-cache\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.786064 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.786017    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tmp-dir\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.786125 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.786078    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tls-certs\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.786204 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.786182    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-home\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.786274 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.786236    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-model-cache\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.786339 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.786324    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tmp-dir\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.788118 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.788092    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-dshm\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.788560 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.788541    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tls-certs\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.793326 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.793296    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5h6k\" (UniqueName: \"kubernetes.io/projected/781a9e72-e05b-4bb1-9a3d-a6b960d37809-kube-api-access-l5h6k\") pod \"auth-test-model-abcdff-kserve-964854fbc-rh6vt\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:32.932288 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:32.932144    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:33.204439 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:33.204414    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"]
Sep 11 19:41:33.206317 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:41:33.206290    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod781a9e72_e05b_4bb1_9a3d_a6b960d37809.slice/crio-0623516eb6f13e65bf5654fe8124f0870a5c46c644db86d028923b867415ef58 WatchSource:0}: Error finding container 0623516eb6f13e65bf5654fe8124f0870a5c46c644db86d028923b867415ef58: Status 404 returned error can't find the container with id 0623516eb6f13e65bf5654fe8124f0870a5c46c644db86d028923b867415ef58
Sep 11 19:41:33.262225 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:33.262195    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt" event={"ID":"781a9e72-e05b-4bb1-9a3d-a6b960d37809","Type":"ContainerStarted","Data":"0623516eb6f13e65bf5654fe8124f0870a5c46c644db86d028923b867415ef58"}
Sep 11 19:41:34.268744 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:34.268711    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt" event={"ID":"781a9e72-e05b-4bb1-9a3d-a6b960d37809","Type":"ContainerStarted","Data":"5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66"}
Sep 11 19:41:34.286416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:34.286368    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt" podStartSLOduration=2.286353204 podStartE2EDuration="2.286353204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:41:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:41:34.284210664 +0000 UTC m=+1398.212835572" watchObservedRunningTime="2026-09-11 19:41:34.286353204 +0000 UTC m=+1398.214978101"
Sep 11 19:41:34.932924 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:34.932890    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:34.932924 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:34.932938    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:34.946310 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:34.946283    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:35.284725 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:35.284643    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:41:40.827571 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:40.827533    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:42.932580 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:42.932539    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:42.933076 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:42.932595    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:42.945334 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:42.945309    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:43.326755 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:43.326677    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:41:46.638481 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:46.638445    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:46.738440 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:46.738412    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:46.964890 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:46.964803    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:47.284577 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:47.284488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:47.464319 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:47.464284    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:49.745226 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:49.745193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:50.401388 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:50.401353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:50.502776 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:50.502744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:50.603200 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:50.603165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:51.211114 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:51.211080    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:51.963177 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:51.963145    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:52.924368 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:52.924317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:53.634312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:53.634276    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:54.338799 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:54.338740    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:54.440776 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:54.440742    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:54.787742 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:54.787704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:54.914536 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:54.914500    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:55.009643 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:55.009616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:55.335342 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:55.335308    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:41:56.178205 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:41:56.178168    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:13.023160 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:13.023079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:13.074184 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:13.074149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:17.270500 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:17.270467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:17.390420 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:17.390368    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:17.483112 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:17.483077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:18.138113 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:18.138081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:18.881529 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:18.881492    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:19.545344 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:19.545308    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:20.448889 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:20.448848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:20.571881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:20.571852    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:21.054835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:21.054802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:21.159576 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:21.159543    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:21.363767 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:21.363731    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:21.848669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:21.848637    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:21.942722 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:21.942685    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:21.971271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:21.971222    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"]
Sep 11 19:42:21.987749 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:21.987719    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"]
Sep 11 19:42:22.055071 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:22.055036    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:22.633148 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:22.633113    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:24.036811 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:24.036780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:24.663557 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:24.663523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:24.763135 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:24.763100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:25.025068 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:25.024980    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:25.833066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:25.833029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:26.641751 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:26.641706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:27.360186 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:27.360151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:28.109550 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:28.109514    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:28.708965 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:28.708936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:28.925791 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:28.925756    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:29.556899 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.556868    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"]
Sep 11 19:42:29.560577 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.560558    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.562534 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.562513    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-491be8-serving-cert\""
Sep 11 19:42:29.573040 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.573009    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"]
Sep 11 19:42:29.616069 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.616032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:29.682919 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.682881    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44570393-a4ea-4b74-8575-1fd612eb4cd9-openshift-service-ca\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.683092 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.682941    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-maas-api-tls\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.683092 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.682996    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-metrics-certs\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.683092 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.683023    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nstr7\" (UniqueName: \"kubernetes.io/projected/44570393-a4ea-4b74-8575-1fd612eb4cd9-kube-api-access-nstr7\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.718465 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.718433    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:29.784554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.784515    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-maas-api-tls\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.784744 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.784569    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-metrics-certs\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.784744 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.784595    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nstr7\" (UniqueName: \"kubernetes.io/projected/44570393-a4ea-4b74-8575-1fd612eb4cd9-kube-api-access-nstr7\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.784744 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.784659    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44570393-a4ea-4b74-8575-1fd612eb4cd9-openshift-service-ca\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.785350 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.785329    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44570393-a4ea-4b74-8575-1fd612eb4cd9-openshift-service-ca\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.787302 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.787274    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-metrics-certs\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.787401 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.787364    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-maas-api-tls\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.792056 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.792035    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nstr7\" (UniqueName: \"kubernetes.io/projected/44570393-a4ea-4b74-8575-1fd612eb4cd9-kube-api-access-nstr7\") pod \"maas-api-e2e-api-b-491be8-8698fcd5db-4gstw\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:29.816576 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.816502    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:29.872395 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:29.872364    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:30.151283 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:30.151135    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"]
Sep 11 19:42:30.152612 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:30.152587    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:30.154581 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:42:30.154555    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44570393_a4ea_4b74_8575_1fd612eb4cd9.slice/crio-b0629cc2f829027b4214cda3baf8b175ebe28112541012dc3b712ead12bddd51 WatchSource:0}: Error finding container b0629cc2f829027b4214cda3baf8b175ebe28112541012dc3b712ead12bddd51: Status 404 returned error can't find the container with id b0629cc2f829027b4214cda3baf8b175ebe28112541012dc3b712ead12bddd51
Sep 11 19:42:30.155912 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:30.155893    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 19:42:30.247039 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:30.247005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:30.547224 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:30.547195    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:30.562641 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:30.562610    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw" event={"ID":"44570393-a4ea-4b74-8575-1fd612eb4cd9","Type":"ContainerStarted","Data":"0c9c00e2ce5c1c91b11705f64d45a8736d40cbd30472ebdbe146b238e340157e"}
Sep 11 19:42:30.562814 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:30.562660    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw" event={"ID":"44570393-a4ea-4b74-8575-1fd612eb4cd9","Type":"ContainerStarted","Data":"b0629cc2f829027b4214cda3baf8b175ebe28112541012dc3b712ead12bddd51"}
Sep 11 19:42:30.562892 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:30.562832    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:30.582492 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:30.582440    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw" podStartSLOduration=1.255010651 podStartE2EDuration="1.582421654s" totalImagesPullingTime="327.411003ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:42:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:42:30.581302773 +0000 UTC m=+1454.509927672" watchObservedRunningTime="2026-09-11 19:42:30.582421654 +0000 UTC m=+1454.511046612"
Sep 11 19:42:31.427609 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:31.427581    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:35.613287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:35.613224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:36.198543 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.198506    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"]
Sep 11 19:42:36.198824 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.198754    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh" podUID="a5203fea-7020-4559-b071-36df336e300b" containerName="maas-api" containerID="cri-o://3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9" gracePeriod=30
Sep 11 19:42:36.231172 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.231146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:36.331651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.331619    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:36.432091 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.431994    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:36.456112 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.456040    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:42:36.497565 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.497538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:36.556002 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.555973    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sptd\" (UniqueName: \"kubernetes.io/projected/a5203fea-7020-4559-b071-36df336e300b-kube-api-access-5sptd\") pod \"a5203fea-7020-4559-b071-36df336e300b\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") "
Sep 11 19:42:36.556186 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.556018    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-metrics-certs\") pod \"a5203fea-7020-4559-b071-36df336e300b\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") "
Sep 11 19:42:36.556186 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.556054    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-maas-api-tls\") pod \"a5203fea-7020-4559-b071-36df336e300b\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") "
Sep 11 19:42:36.556186 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.556172    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5203fea-7020-4559-b071-36df336e300b-openshift-service-ca\") pod \"a5203fea-7020-4559-b071-36df336e300b\" (UID: \"a5203fea-7020-4559-b071-36df336e300b\") "
Sep 11 19:42:36.556529 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.556493    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5203fea-7020-4559-b071-36df336e300b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a5203fea-7020-4559-b071-36df336e300b" (UID: "a5203fea-7020-4559-b071-36df336e300b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:42:36.558573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.558548    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a5203fea-7020-4559-b071-36df336e300b" (UID: "a5203fea-7020-4559-b071-36df336e300b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:42:36.558822 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.558804    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5203fea-7020-4559-b071-36df336e300b-kube-api-access-5sptd" (OuterVolumeSpecName: "kube-api-access-5sptd") pod "a5203fea-7020-4559-b071-36df336e300b" (UID: "a5203fea-7020-4559-b071-36df336e300b"). InnerVolumeSpecName "kube-api-access-5sptd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:42:36.558876 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.558826    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a5203fea-7020-4559-b071-36df336e300b" (UID: "a5203fea-7020-4559-b071-36df336e300b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:42:36.575689 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.575664    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:42:36.598830 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.598782    2583 generic.go:356] "Generic (PLEG): container finished" podID="a5203fea-7020-4559-b071-36df336e300b" containerID="3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9" exitCode=0
Sep 11 19:42:36.599009 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.598865    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh" event={"ID":"a5203fea-7020-4559-b071-36df336e300b","Type":"ContainerDied","Data":"3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9"}
Sep 11 19:42:36.599009 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.598894    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"
Sep 11 19:42:36.599009 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.598915    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh" event={"ID":"a5203fea-7020-4559-b071-36df336e300b","Type":"ContainerDied","Data":"d8b2127f0664cff7d885992545280bed14f4b5043dc7dac2bf101e76d7671082"}
Sep 11 19:42:36.599009 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.598945    2583 scope.go:117] "RemoveContainer" containerID="3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9"
Sep 11 19:42:36.611517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.611499    2583 scope.go:117] "RemoveContainer" containerID="3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9"
Sep 11 19:42:36.611826 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:42:36.611805    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9\": container with ID starting with 3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9 not found: ID does not exist" containerID="3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9"
Sep 11 19:42:36.611888 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.611837    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9"} err="failed to get container status \"3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9\": rpc error: code = NotFound desc = could not find container \"3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9\": container with ID starting with 3609a0de38980355db1993b18b52a58cbd7b261eea249e22b56c66fc408791e9 not found: ID does not exist"
Sep 11 19:42:36.624588 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.624540    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"]
Sep 11 19:42:36.628318 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.628291    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1fc808-7bb446f556-hg5kh"]
Sep 11 19:42:36.657528 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.657495    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sptd\" (UniqueName: \"kubernetes.io/projected/a5203fea-7020-4559-b071-36df336e300b-kube-api-access-5sptd\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:36.657731 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.657544    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:36.657731 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.657559    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5203fea-7020-4559-b071-36df336e300b-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:36.657731 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.657569    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5203fea-7020-4559-b071-36df336e300b-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:36.837280 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.837154    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:36.936062 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.936031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:36.992030 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:36.991990    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q" podUID="bb451d72-d2c8-4796-979e-4a00420ab008" containerName="main" containerID="cri-o://238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa" gracePeriod=45
Sep 11 19:42:37.007962 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.007923    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7" podUID="57932cd5-9b5f-48e4-b747-572df168c661" containerName="main" containerID="cri-o://6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e" gracePeriod=45
Sep 11 19:42:37.288303 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.288275    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:42:37.292026 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.292003    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:42:37.364615 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.364526    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-tmp-dir\") pod \"bb451d72-d2c8-4796-979e-4a00420ab008\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") "
Sep 11 19:42:37.364615 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.364587    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-dshm\") pod \"bb451d72-d2c8-4796-979e-4a00420ab008\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") "
Sep 11 19:42:37.364841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.364616    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-tmp-dir\") pod \"57932cd5-9b5f-48e4-b747-572df168c661\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") "
Sep 11 19:42:37.364841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.364649    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb451d72-d2c8-4796-979e-4a00420ab008-tls-certs\") pod \"bb451d72-d2c8-4796-979e-4a00420ab008\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") "
Sep 11 19:42:37.364841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.364687    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff88t\" (UniqueName: \"kubernetes.io/projected/57932cd5-9b5f-48e4-b747-572df168c661-kube-api-access-ff88t\") pod \"57932cd5-9b5f-48e4-b747-572df168c661\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") "
Sep 11 19:42:37.364841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.364801    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-model-cache\") pod \"bb451d72-d2c8-4796-979e-4a00420ab008\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") "
Sep 11 19:42:37.364841 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.364831    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-home\") pod \"bb451d72-d2c8-4796-979e-4a00420ab008\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") "
Sep 11 19:42:37.365097 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.364894    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-model-cache\") pod \"57932cd5-9b5f-48e4-b747-572df168c661\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") "
Sep 11 19:42:37.365097 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.364923    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-home\") pod \"57932cd5-9b5f-48e4-b747-572df168c661\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") "
Sep 11 19:42:37.365097 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.364945    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "57932cd5-9b5f-48e4-b747-572df168c661" (UID: "57932cd5-9b5f-48e4-b747-572df168c661"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:42:37.365097 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.364970    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4j2h8\" (UniqueName: \"kubernetes.io/projected/bb451d72-d2c8-4796-979e-4a00420ab008-kube-api-access-4j2h8\") pod \"bb451d72-d2c8-4796-979e-4a00420ab008\" (UID: \"bb451d72-d2c8-4796-979e-4a00420ab008\") "
Sep 11 19:42:37.365097 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.365043    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-dshm\") pod \"57932cd5-9b5f-48e4-b747-572df168c661\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") "
Sep 11 19:42:37.365355 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.365116    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57932cd5-9b5f-48e4-b747-572df168c661-tls-certs\") pod \"57932cd5-9b5f-48e4-b747-572df168c661\" (UID: \"57932cd5-9b5f-48e4-b747-572df168c661\") "
Sep 11 19:42:37.365408 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.365379    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-home" (OuterVolumeSpecName: "home") pod "bb451d72-d2c8-4796-979e-4a00420ab008" (UID: "bb451d72-d2c8-4796-979e-4a00420ab008"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:42:37.365818 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.365598    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-tmp-dir\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.365818 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.365775    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-home\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.365818 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.365659    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bb451d72-d2c8-4796-979e-4a00420ab008" (UID: "bb451d72-d2c8-4796-979e-4a00420ab008"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:42:37.366037 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.365911    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-home" (OuterVolumeSpecName: "home") pod "57932cd5-9b5f-48e4-b747-572df168c661" (UID: "57932cd5-9b5f-48e4-b747-572df168c661"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:42:37.366037 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.365966    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-model-cache" (OuterVolumeSpecName: "model-cache") pod "bb451d72-d2c8-4796-979e-4a00420ab008" (UID: "bb451d72-d2c8-4796-979e-4a00420ab008"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:42:37.366037 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.365996    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-model-cache" (OuterVolumeSpecName: "model-cache") pod "57932cd5-9b5f-48e4-b747-572df168c661" (UID: "57932cd5-9b5f-48e4-b747-572df168c661"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:42:37.367824 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.367788    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-dshm" (OuterVolumeSpecName: "dshm") pod "bb451d72-d2c8-4796-979e-4a00420ab008" (UID: "bb451d72-d2c8-4796-979e-4a00420ab008"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:42:37.367940 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.367876    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb451d72-d2c8-4796-979e-4a00420ab008-kube-api-access-4j2h8" (OuterVolumeSpecName: "kube-api-access-4j2h8") pod "bb451d72-d2c8-4796-979e-4a00420ab008" (UID: "bb451d72-d2c8-4796-979e-4a00420ab008"). InnerVolumeSpecName "kube-api-access-4j2h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:42:37.368044 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.368019    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57932cd5-9b5f-48e4-b747-572df168c661-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "57932cd5-9b5f-48e4-b747-572df168c661" (UID: "57932cd5-9b5f-48e4-b747-572df168c661"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:42:37.368674 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.368650    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57932cd5-9b5f-48e4-b747-572df168c661-kube-api-access-ff88t" (OuterVolumeSpecName: "kube-api-access-ff88t") pod "57932cd5-9b5f-48e4-b747-572df168c661" (UID: "57932cd5-9b5f-48e4-b747-572df168c661"). InnerVolumeSpecName "kube-api-access-ff88t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:42:37.368891 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.368867    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb451d72-d2c8-4796-979e-4a00420ab008-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bb451d72-d2c8-4796-979e-4a00420ab008" (UID: "bb451d72-d2c8-4796-979e-4a00420ab008"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:42:37.369049 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.369032    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-dshm" (OuterVolumeSpecName: "dshm") pod "57932cd5-9b5f-48e4-b747-572df168c661" (UID: "57932cd5-9b5f-48e4-b747-572df168c661"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:42:37.467273 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.467211    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-model-cache\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.467273 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.467272    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-home\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.467484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.467288    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4j2h8\" (UniqueName: \"kubernetes.io/projected/bb451d72-d2c8-4796-979e-4a00420ab008-kube-api-access-4j2h8\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.467484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.467303    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57932cd5-9b5f-48e4-b747-572df168c661-dshm\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.467484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.467315    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57932cd5-9b5f-48e4-b747-572df168c661-tls-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.467484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.467326    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-tmp-dir\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.467484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.467336    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-dshm\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.467484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.467346    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb451d72-d2c8-4796-979e-4a00420ab008-tls-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.467484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.467357    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff88t\" (UniqueName: \"kubernetes.io/projected/57932cd5-9b5f-48e4-b747-572df168c661-kube-api-access-ff88t\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.467484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.467367    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb451d72-d2c8-4796-979e-4a00420ab008-model-cache\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:42:37.606571 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.606531    2583 generic.go:356] "Generic (PLEG): container finished" podID="bb451d72-d2c8-4796-979e-4a00420ab008" containerID="238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa" exitCode=0
Sep 11 19:42:37.606758 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.606580    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"
Sep 11 19:42:37.606758 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.606619    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q" event={"ID":"bb451d72-d2c8-4796-979e-4a00420ab008","Type":"ContainerDied","Data":"238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa"}
Sep 11 19:42:37.606758 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.606672    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q" event={"ID":"bb451d72-d2c8-4796-979e-4a00420ab008","Type":"ContainerDied","Data":"33435ad132ac6d63e7a1dfa1ac4b73c2eff6d58de4182fe553fbc8f056af97cc"}
Sep 11 19:42:37.606758 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.606698    2583 scope.go:117] "RemoveContainer" containerID="238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa"
Sep 11 19:42:37.608147 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.608122    2583 generic.go:356] "Generic (PLEG): container finished" podID="57932cd5-9b5f-48e4-b747-572df168c661" containerID="6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e" exitCode=0
Sep 11 19:42:37.608276 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.608176    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"
Sep 11 19:42:37.608276 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.608185    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7" event={"ID":"57932cd5-9b5f-48e4-b747-572df168c661","Type":"ContainerDied","Data":"6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e"}
Sep 11 19:42:37.608276 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.608263    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7" event={"ID":"57932cd5-9b5f-48e4-b747-572df168c661","Type":"ContainerDied","Data":"c1ab690454979eb9cd57307b755741685f230ece965de7f72a7eae669030218d"}
Sep 11 19:42:37.617000 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.616985    2583 scope.go:117] "RemoveContainer" containerID="238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa"
Sep 11 19:42:37.617309 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:42:37.617290    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa\": container with ID starting with 238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa not found: ID does not exist" containerID="238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa"
Sep 11 19:42:37.617382 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.617316    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa"} err="failed to get container status \"238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa\": rpc error: code = NotFound desc = could not find container \"238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa\": container with ID starting with 238d663ac3d7f6b88825aa153a905cd5b9b22a8998090565f31bbce6927452fa not found: ID does not exist"
Sep 11 19:42:37.617382 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.617342    2583 scope.go:117] "RemoveContainer" containerID="6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e"
Sep 11 19:42:37.627429 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.627406    2583 scope.go:117] "RemoveContainer" containerID="6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e"
Sep 11 19:42:37.627764 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:42:37.627737    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e\": container with ID starting with 6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e not found: ID does not exist" containerID="6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e"
Sep 11 19:42:37.627810 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.627758    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e"} err="failed to get container status \"6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e\": rpc error: code = NotFound desc = could not find container \"6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e\": container with ID starting with 6a30258ebc98a52ce196724adbc91d3e9c7b596ceb125378ab9bf997deb0c86e not found: ID does not exist"
Sep 11 19:42:37.629354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.629334    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"]
Sep 11 19:42:37.631195 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.631176    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/embedding-w0-1fc808-kserve-6fc6db6d84-2hb7q"]
Sep 11 19:42:37.640378 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.640343    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"]
Sep 11 19:42:37.645351 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.645329    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-1fc808/simulator-w0-1fc808-kserve-74556bbcf7-9zdw7"]
Sep 11 19:42:37.746569 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:37.746532    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:38.412940 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:38.412909    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:38.546594 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:38.546558    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57932cd5-9b5f-48e4-b747-572df168c661" path="/var/lib/kubelet/pods/57932cd5-9b5f-48e4-b747-572df168c661/volumes"
Sep 11 19:42:38.546962 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:38.546949    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5203fea-7020-4559-b071-36df336e300b" path="/var/lib/kubelet/pods/a5203fea-7020-4559-b071-36df336e300b/volumes"
Sep 11 19:42:38.547311 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:38.547299    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb451d72-d2c8-4796-979e-4a00420ab008" path="/var/lib/kubelet/pods/bb451d72-d2c8-4796-979e-4a00420ab008/volumes"
Sep 11 19:42:39.153072 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:39.153039    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:39.836283 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:39.836206    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:40.513850 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:40.513816    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:41.402425 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:41.402390    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:42.233394 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:42.233358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:42.927050 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:42.927014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:43.024506 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:43.024475    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:43.133403 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:43.133369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:43.209152 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:43.209060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:43.955211 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:43.955174    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:44.469115 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:44.469069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:44.840400 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:44.840294    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:45.542840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:45.542802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:46.444659 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:46.444626    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:47.142034 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:47.141985    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:47.862072 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:47.862035    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:48.627235 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:48.627206    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:49.282808 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:49.282773    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:50.004659 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:50.004607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:50.950065 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:50.950031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:51.596789 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:51.596756    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:52.255263 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:52.255192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:57.427441 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:57.427408    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:58.244067 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:58.244031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:58.344175 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:58.344146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:42:59.494398 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:42:59.494366    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:03.034358 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:03.034330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:03.104519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:03.104490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:03.166426 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:03.166389    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:03.224037 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:03.223996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:03.293288 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:03.293172    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:03.554044 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:03.553944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:03.655364 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:03.655327    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:04.333371 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:04.333333    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:05.129029 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:05.128992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:06.022584 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:06.022541    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:06.944377 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:06.944329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:07.632077 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:07.632044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:09.472624 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:09.472593    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:09.545391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:09.545346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:09.782643 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:09.782522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:10.558867 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:10.558829    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:11.383917 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:11.383880    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:12.118445 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:12.118408    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:12.779444 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:12.779404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:13.530619 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:13.530585    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:14.143702 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:14.143668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:14.866698 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:14.866611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:25.654573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:25.654532    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"]
Sep 11 19:43:25.655026 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:25.654733    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw" podUID="44570393-a4ea-4b74-8575-1fd612eb4cd9" containerName="maas-api" containerID="cri-o://0c9c00e2ce5c1c91b11705f64d45a8736d40cbd30472ebdbe146b238e340157e" gracePeriod=30
Sep 11 19:43:25.682427 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:25.682393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:25.754986 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:25.754950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:25.816109 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:25.816075    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:25.879566 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:25.879531    2583 generic.go:356] "Generic (PLEG): container finished" podID="44570393-a4ea-4b74-8575-1fd612eb4cd9" containerID="0c9c00e2ce5c1c91b11705f64d45a8736d40cbd30472ebdbe146b238e340157e" exitCode=0
Sep 11 19:43:25.879734 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:25.879578    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw" event={"ID":"44570393-a4ea-4b74-8575-1fd612eb4cd9","Type":"ContainerDied","Data":"0c9c00e2ce5c1c91b11705f64d45a8736d40cbd30472ebdbe146b238e340157e"}
Sep 11 19:43:25.883304 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:25.883271    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:25.916083 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:25.916026    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:43:25.947312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:25.947268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:26.041974 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.041945    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-maas-api-tls\") pod \"44570393-a4ea-4b74-8575-1fd612eb4cd9\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") "
Sep 11 19:43:26.042167 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.042112    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nstr7\" (UniqueName: \"kubernetes.io/projected/44570393-a4ea-4b74-8575-1fd612eb4cd9-kube-api-access-nstr7\") pod \"44570393-a4ea-4b74-8575-1fd612eb4cd9\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") "
Sep 11 19:43:26.042167 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.042140    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44570393-a4ea-4b74-8575-1fd612eb4cd9-openshift-service-ca\") pod \"44570393-a4ea-4b74-8575-1fd612eb4cd9\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") "
Sep 11 19:43:26.042167 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.042162    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-metrics-certs\") pod \"44570393-a4ea-4b74-8575-1fd612eb4cd9\" (UID: \"44570393-a4ea-4b74-8575-1fd612eb4cd9\") "
Sep 11 19:43:26.042556 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.042502    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44570393-a4ea-4b74-8575-1fd612eb4cd9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "44570393-a4ea-4b74-8575-1fd612eb4cd9" (UID: "44570393-a4ea-4b74-8575-1fd612eb4cd9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:43:26.044517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.044491    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "44570393-a4ea-4b74-8575-1fd612eb4cd9" (UID: "44570393-a4ea-4b74-8575-1fd612eb4cd9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:43:26.044947 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.044928    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "44570393-a4ea-4b74-8575-1fd612eb4cd9" (UID: "44570393-a4ea-4b74-8575-1fd612eb4cd9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:43:26.045053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.044964    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44570393-a4ea-4b74-8575-1fd612eb4cd9-kube-api-access-nstr7" (OuterVolumeSpecName: "kube-api-access-nstr7") pod "44570393-a4ea-4b74-8575-1fd612eb4cd9" (UID: "44570393-a4ea-4b74-8575-1fd612eb4cd9"). InnerVolumeSpecName "kube-api-access-nstr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:43:26.142908 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.142871    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nstr7\" (UniqueName: \"kubernetes.io/projected/44570393-a4ea-4b74-8575-1fd612eb4cd9-kube-api-access-nstr7\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:43:26.142908 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.142900    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44570393-a4ea-4b74-8575-1fd612eb4cd9-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:43:26.142908 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.142915    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:43:26.143146 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.142927    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44570393-a4ea-4b74-8575-1fd612eb4cd9-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:43:26.241305 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.241195    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:26.306696 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.306658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:26.886346 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.886301    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw" event={"ID":"44570393-a4ea-4b74-8575-1fd612eb4cd9","Type":"ContainerDied","Data":"b0629cc2f829027b4214cda3baf8b175ebe28112541012dc3b712ead12bddd51"}
Sep 11 19:43:26.886346 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.886345    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"
Sep 11 19:43:26.886830 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.886373    2583 scope.go:117] "RemoveContainer" containerID="0c9c00e2ce5c1c91b11705f64d45a8736d40cbd30472ebdbe146b238e340157e"
Sep 11 19:43:26.903286 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.903222    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"]
Sep 11 19:43:26.906358 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:26.906333    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-491be8-8698fcd5db-4gstw"]
Sep 11 19:43:27.046068 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:27.046026    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:27.842646 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:27.842613    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:28.516096 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:28.516063    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:28.546588 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:28.546556    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44570393-a4ea-4b74-8575-1fd612eb4cd9" path="/var/lib/kubelet/pods/44570393-a4ea-4b74-8575-1fd612eb4cd9/volumes"
Sep 11 19:43:29.310312 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:29.310234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:31.942760 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:31.942729    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:32.394793 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:32.394759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:32.455954 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:32.455925    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:32.753624 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:32.753519    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:33.262962 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:33.262838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:34.127014 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:34.126984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:34.891927 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:34.891890    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:35.535265 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:35.535214    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:36.229505 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:36.229473    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:36.342994 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:36.342957    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:37.298059 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:37.298019    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:37.842148 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:37.842117    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:43.644640 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.644608    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"]
Sep 11 19:43:43.645027 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.644863    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6" podUID="5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3" containerName="maas-api" containerID="cri-o://f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb" gracePeriod=30
Sep 11 19:43:43.673138 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.673110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:43.737608 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.737568    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:43.840718 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.840678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:43.895403 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.895345    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:43:43.901733 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.901684    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:43.974170 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.974131    2583 generic.go:356] "Generic (PLEG): container finished" podID="5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3" containerID="f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb" exitCode=0
Sep 11 19:43:43.974381 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.974195    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"
Sep 11 19:43:43.974381 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.974229    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6" event={"ID":"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3","Type":"ContainerDied","Data":"f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb"}
Sep 11 19:43:43.974381 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.974319    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6" event={"ID":"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3","Type":"ContainerDied","Data":"0e47941b9897c2c85ae1ca58306b849aca383865e6e041ff621d7110b14b2d79"}
Sep 11 19:43:43.974381 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.974343    2583 scope.go:117] "RemoveContainer" containerID="f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb"
Sep 11 19:43:43.983297 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.983277    2583 scope.go:117] "RemoveContainer" containerID="f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb"
Sep 11 19:43:43.983577 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:43:43.983559    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb\": container with ID starting with f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb not found: ID does not exist" containerID="f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb"
Sep 11 19:43:43.983650 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:43.983586    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb"} err="failed to get container status \"f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb\": rpc error: code = NotFound desc = could not find container \"f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb\": container with ID starting with f5ee2deafa46f1327d1e5618eddb925a29537d9abdf52125527a1852217d40eb not found: ID does not exist"
Sep 11 19:43:44.021301 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.021267    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-metrics-certs\") pod \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") "
Sep 11 19:43:44.021481 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.021378    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-maas-api-tls\") pod \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") "
Sep 11 19:43:44.021481 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.021450    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-openshift-service-ca\") pod \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") "
Sep 11 19:43:44.021608 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.021533    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8dgv\" (UniqueName: \"kubernetes.io/projected/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-kube-api-access-q8dgv\") pod \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\" (UID: \"5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3\") "
Sep 11 19:43:44.021798 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.021776    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3" (UID: "5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:43:44.021885 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.021868    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:43:44.023681 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.023653    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3" (UID: "5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:43:44.023786 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.023710    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-kube-api-access-q8dgv" (OuterVolumeSpecName: "kube-api-access-q8dgv") pod "5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3" (UID: "5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3"). InnerVolumeSpecName "kube-api-access-q8dgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:43:44.023786 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.023715    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3" (UID: "5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:43:44.123181 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.123146    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:43:44.123181 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.123175    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8dgv\" (UniqueName: \"kubernetes.io/projected/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-kube-api-access-q8dgv\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:43:44.123181 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.123184    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:43:44.191513 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.191414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:44.294745 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.294712    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"]
Sep 11 19:43:44.296226 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.296203    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:44.298944 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.298922    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5bf35-5b76b66cc8-x25g6"]
Sep 11 19:43:44.546656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:44.546572    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3" path="/var/lib/kubelet/pods/5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3/volumes"
Sep 11 19:43:45.038914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:45.038878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:45.946081 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:45.946052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:46.636686 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:46.636653    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:49.972601 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:49.972569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:50.031452 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:50.031423    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:50.253041 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:50.252964    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:50.846763 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:50.846726    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:51.581931 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:51.581897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:52.323832 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:52.323796    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:53.029598 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:53.029560    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:43:53.673918 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:43:53.673872    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:01.084521 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.084490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:01.118220 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.118179    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"]
Sep 11 19:44:01.118492 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.118445    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7" podUID="01b47d19-c133-4339-820e-56a8521a4994" containerName="payload-processing" containerID="cri-o://77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54" gracePeriod=30
Sep 11 19:44:01.122229 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.122196    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"]
Sep 11 19:44:01.122973 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.122916    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9" podUID="8dd0b9d1-a040-4b61-9bac-30413990044e" containerName="payload-pre-processing" containerID="cri-o://477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f" gracePeriod=30
Sep 11 19:44:01.159853 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.159826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:01.259557 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.259513    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:01.336389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.336276    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:01.381937 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.381908    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:44:01.416035 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.416009    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:44:01.490550 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.490518    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01b47d19-c133-4339-820e-56a8521a4994-plugins-config-volume\") pod \"01b47d19-c133-4339-820e-56a8521a4994\" (UID: \"01b47d19-c133-4339-820e-56a8521a4994\") "
Sep 11 19:44:01.490767 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.490562    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp9dm\" (UniqueName: \"kubernetes.io/projected/01b47d19-c133-4339-820e-56a8521a4994-kube-api-access-fp9dm\") pod \"01b47d19-c133-4339-820e-56a8521a4994\" (UID: \"01b47d19-c133-4339-820e-56a8521a4994\") "
Sep 11 19:44:01.490924 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.490898    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b47d19-c133-4339-820e-56a8521a4994-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "01b47d19-c133-4339-820e-56a8521a4994" (UID: "01b47d19-c133-4339-820e-56a8521a4994"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:44:01.493055 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.493023    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01b47d19-c133-4339-820e-56a8521a4994-kube-api-access-fp9dm" (OuterVolumeSpecName: "kube-api-access-fp9dm") pod "01b47d19-c133-4339-820e-56a8521a4994" (UID: "01b47d19-c133-4339-820e-56a8521a4994"). InnerVolumeSpecName "kube-api-access-fp9dm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:44:01.591692 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.591599    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dd0b9d1-a040-4b61-9bac-30413990044e-plugins-config-volume\") pod \"8dd0b9d1-a040-4b61-9bac-30413990044e\" (UID: \"8dd0b9d1-a040-4b61-9bac-30413990044e\") "
Sep 11 19:44:01.591692 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.591691    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mks55\" (UniqueName: \"kubernetes.io/projected/8dd0b9d1-a040-4b61-9bac-30413990044e-kube-api-access-mks55\") pod \"8dd0b9d1-a040-4b61-9bac-30413990044e\" (UID: \"8dd0b9d1-a040-4b61-9bac-30413990044e\") "
Sep 11 19:44:01.591941 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.591873    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01b47d19-c133-4339-820e-56a8521a4994-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:01.591941 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.591883    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp9dm\" (UniqueName: \"kubernetes.io/projected/01b47d19-c133-4339-820e-56a8521a4994-kube-api-access-fp9dm\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:01.592050 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.591971    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dd0b9d1-a040-4b61-9bac-30413990044e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8dd0b9d1-a040-4b61-9bac-30413990044e" (UID: "8dd0b9d1-a040-4b61-9bac-30413990044e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:44:01.593894 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.593863    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd0b9d1-a040-4b61-9bac-30413990044e-kube-api-access-mks55" (OuterVolumeSpecName: "kube-api-access-mks55") pod "8dd0b9d1-a040-4b61-9bac-30413990044e" (UID: "8dd0b9d1-a040-4b61-9bac-30413990044e"). InnerVolumeSpecName "kube-api-access-mks55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:44:01.626579 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.626545    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:01.693040 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.693008    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mks55\" (UniqueName: \"kubernetes.io/projected/8dd0b9d1-a040-4b61-9bac-30413990044e-kube-api-access-mks55\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:01.693040 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.693036    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dd0b9d1-a040-4b61-9bac-30413990044e-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:01.774145 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:01.774114    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:02.065356 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.065322    2583 generic.go:356] "Generic (PLEG): container finished" podID="01b47d19-c133-4339-820e-56a8521a4994" containerID="77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54" exitCode=0
Sep 11 19:44:02.065537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.065369    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"
Sep 11 19:44:02.065537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.065421    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7" event={"ID":"01b47d19-c133-4339-820e-56a8521a4994","Type":"ContainerDied","Data":"77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54"}
Sep 11 19:44:02.065537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.065477    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7" event={"ID":"01b47d19-c133-4339-820e-56a8521a4994","Type":"ContainerDied","Data":"6fab17796a71f8f2ff6010455341470d105826f42c29a2e6285350e4f7b8681f"}
Sep 11 19:44:02.065537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.065508    2583 scope.go:117] "RemoveContainer" containerID="77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54"
Sep 11 19:44:02.066772 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.066753    2583 generic.go:356] "Generic (PLEG): container finished" podID="8dd0b9d1-a040-4b61-9bac-30413990044e" containerID="477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f" exitCode=0
Sep 11 19:44:02.066869 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.066789    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"
Sep 11 19:44:02.066869 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.066807    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9" event={"ID":"8dd0b9d1-a040-4b61-9bac-30413990044e","Type":"ContainerDied","Data":"477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f"}
Sep 11 19:44:02.066869 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.066838    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9" event={"ID":"8dd0b9d1-a040-4b61-9bac-30413990044e","Type":"ContainerDied","Data":"71d324bc4d5e877c7b33ecbef8359fd54fb2cbab14e066e3718231161f4501ca"}
Sep 11 19:44:02.076064 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.076043    2583 scope.go:117] "RemoveContainer" containerID="77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54"
Sep 11 19:44:02.076427 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:44:02.076397    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54\": container with ID starting with 77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54 not found: ID does not exist" containerID="77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54"
Sep 11 19:44:02.076517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.076437    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54"} err="failed to get container status \"77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54\": rpc error: code = NotFound desc = could not find container \"77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54\": container with ID starting with 77e217a1e9e0681ce616a7b83cde36bda89495f9188d264a11c519885d604b54 not found: ID does not exist"
Sep 11 19:44:02.076517 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.076461    2583 scope.go:117] "RemoveContainer" containerID="477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f"
Sep 11 19:44:02.084979 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.084954    2583 scope.go:117] "RemoveContainer" containerID="477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f"
Sep 11 19:44:02.085364 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:44:02.085280    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f\": container with ID starting with 477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f not found: ID does not exist" containerID="477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f"
Sep 11 19:44:02.085364 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.085305    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f"} err="failed to get container status \"477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f\": rpc error: code = NotFound desc = could not find container \"477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f\": container with ID starting with 477efa1023505d4b5e2de67c1238d2b804d468484d53bc96e8aa009e1c6d244f not found: ID does not exist"
Sep 11 19:44:02.089172 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.089147    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"]
Sep 11 19:44:02.095458 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.095437    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b20815-d4f67c555-dt5v7"]
Sep 11 19:44:02.104717 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.104694    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"]
Sep 11 19:44:02.108208 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.108187    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b20815-6df87cb8c8-hxmp9"]
Sep 11 19:44:02.540596 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.540561    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:02.545861 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.545833    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01b47d19-c133-4339-820e-56a8521a4994" path="/var/lib/kubelet/pods/01b47d19-c133-4339-820e-56a8521a4994/volumes"
Sep 11 19:44:02.546209 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:02.546196    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dd0b9d1-a040-4b61-9bac-30413990044e" path="/var/lib/kubelet/pods/8dd0b9d1-a040-4b61-9bac-30413990044e/volumes"
Sep 11 19:44:03.233639 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:03.233594    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:06.519992 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:06.519954    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"]
Sep 11 19:44:06.537876 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:06.537840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:07.396472 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:07.396439    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"]
Sep 11 19:44:07.416895 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:07.416862    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:07.678404 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:07.678301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:07.780378 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:07.780343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:07.920635 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:07.920597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:18.236677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:18.236632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:44:21.540350 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.540287    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8" podUID="c83f28ea-8770-4fd1-9708-9dea39965fe3" containerName="main" containerID="cri-o://446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5" gracePeriod=45
Sep 11 19:44:21.792495 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.792434    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:44:21.890022 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.889982    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-home\") pod \"c83f28ea-8770-4fd1-9708-9dea39965fe3\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") "
Sep 11 19:44:21.890022 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.890028    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-tmp-dir\") pod \"c83f28ea-8770-4fd1-9708-9dea39965fe3\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") "
Sep 11 19:44:21.890321 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.890076    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-dshm\") pod \"c83f28ea-8770-4fd1-9708-9dea39965fe3\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") "
Sep 11 19:44:21.890321 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.890128    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-model-cache\") pod \"c83f28ea-8770-4fd1-9708-9dea39965fe3\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") "
Sep 11 19:44:21.890321 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.890176    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c83f28ea-8770-4fd1-9708-9dea39965fe3-tls-certs\") pod \"c83f28ea-8770-4fd1-9708-9dea39965fe3\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") "
Sep 11 19:44:21.890321 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.890284    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpqml\" (UniqueName: \"kubernetes.io/projected/c83f28ea-8770-4fd1-9708-9dea39965fe3-kube-api-access-qpqml\") pod \"c83f28ea-8770-4fd1-9708-9dea39965fe3\" (UID: \"c83f28ea-8770-4fd1-9708-9dea39965fe3\") "
Sep 11 19:44:21.890513 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.890305    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-home" (OuterVolumeSpecName: "home") pod "c83f28ea-8770-4fd1-9708-9dea39965fe3" (UID: "c83f28ea-8770-4fd1-9708-9dea39965fe3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:44:21.890513 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.890350    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c83f28ea-8770-4fd1-9708-9dea39965fe3" (UID: "c83f28ea-8770-4fd1-9708-9dea39965fe3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:44:21.890513 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.890435    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-model-cache" (OuterVolumeSpecName: "model-cache") pod "c83f28ea-8770-4fd1-9708-9dea39965fe3" (UID: "c83f28ea-8770-4fd1-9708-9dea39965fe3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:44:21.890660 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.890615    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-home\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:21.890660 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.890635    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-tmp-dir\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:21.890660 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.890646    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-model-cache\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:21.892558 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.892531    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-dshm" (OuterVolumeSpecName: "dshm") pod "c83f28ea-8770-4fd1-9708-9dea39965fe3" (UID: "c83f28ea-8770-4fd1-9708-9dea39965fe3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:44:21.892684 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.892559    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c83f28ea-8770-4fd1-9708-9dea39965fe3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c83f28ea-8770-4fd1-9708-9dea39965fe3" (UID: "c83f28ea-8770-4fd1-9708-9dea39965fe3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:44:21.892684 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.892584    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c83f28ea-8770-4fd1-9708-9dea39965fe3-kube-api-access-qpqml" (OuterVolumeSpecName: "kube-api-access-qpqml") pod "c83f28ea-8770-4fd1-9708-9dea39965fe3" (UID: "c83f28ea-8770-4fd1-9708-9dea39965fe3"). InnerVolumeSpecName "kube-api-access-qpqml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:44:21.991350 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.991312    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c83f28ea-8770-4fd1-9708-9dea39965fe3-tls-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:21.991350 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.991343    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpqml\" (UniqueName: \"kubernetes.io/projected/c83f28ea-8770-4fd1-9708-9dea39965fe3-kube-api-access-qpqml\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:21.991350 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:21.991354    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c83f28ea-8770-4fd1-9708-9dea39965fe3-dshm\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:22.169878 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.169844    2583 generic.go:356] "Generic (PLEG): container finished" podID="c83f28ea-8770-4fd1-9708-9dea39965fe3" containerID="446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5" exitCode=0
Sep 11 19:44:22.170060 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.169885    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"
Sep 11 19:44:22.170060 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.169915    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8" event={"ID":"c83f28ea-8770-4fd1-9708-9dea39965fe3","Type":"ContainerDied","Data":"446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5"}
Sep 11 19:44:22.170060 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.169952    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8" event={"ID":"c83f28ea-8770-4fd1-9708-9dea39965fe3","Type":"ContainerDied","Data":"c326c7b3125a23b539319a2c1db0374eb36948235266277e1c339b76e630899c"}
Sep 11 19:44:22.170060 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.169970    2583 scope.go:117] "RemoveContainer" containerID="446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5"
Sep 11 19:44:22.181040 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.181016    2583 scope.go:117] "RemoveContainer" containerID="446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5"
Sep 11 19:44:22.181606 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:44:22.181581    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5\": container with ID starting with 446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5 not found: ID does not exist" containerID="446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5"
Sep 11 19:44:22.181717 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.181623    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5"} err="failed to get container status \"446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5\": rpc error: code = NotFound desc = could not find container \"446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5\": container with ID starting with 446a8165f9bd4117b41c965cadb266144ddf3082bb28fe143536419aaafa2ab5 not found: ID does not exist"
Sep 11 19:44:22.191254 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.191221    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"]
Sep 11 19:44:22.195223 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.195201    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/auth-test-model-19859a-kserve-f6bd947c9-j2cm8"]
Sep 11 19:44:22.416833 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.416787    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt" podUID="781a9e72-e05b-4bb1-9a3d-a6b960d37809" containerName="main" containerID="cri-o://5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66" gracePeriod=45
Sep 11 19:44:22.547444 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.547410    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c83f28ea-8770-4fd1-9708-9dea39965fe3" path="/var/lib/kubelet/pods/c83f28ea-8770-4fd1-9708-9dea39965fe3/volumes"
Sep 11 19:44:22.678802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.678742    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:44:22.797899 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.797860    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5h6k\" (UniqueName: \"kubernetes.io/projected/781a9e72-e05b-4bb1-9a3d-a6b960d37809-kube-api-access-l5h6k\") pod \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") "
Sep 11 19:44:22.797899 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.797903    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-dshm\") pod \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") "
Sep 11 19:44:22.798130 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.797972    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tmp-dir\") pod \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") "
Sep 11 19:44:22.798130 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.797994    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-model-cache\") pod \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") "
Sep 11 19:44:22.798130 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.798035    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tls-certs\") pod \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") "
Sep 11 19:44:22.798130 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.798079    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-home\") pod \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\" (UID: \"781a9e72-e05b-4bb1-9a3d-a6b960d37809\") "
Sep 11 19:44:22.798383 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.798269    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "781a9e72-e05b-4bb1-9a3d-a6b960d37809" (UID: "781a9e72-e05b-4bb1-9a3d-a6b960d37809"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:44:22.798383 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.798325    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-model-cache" (OuterVolumeSpecName: "model-cache") pod "781a9e72-e05b-4bb1-9a3d-a6b960d37809" (UID: "781a9e72-e05b-4bb1-9a3d-a6b960d37809"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:44:22.798502 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.798429    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-home" (OuterVolumeSpecName: "home") pod "781a9e72-e05b-4bb1-9a3d-a6b960d37809" (UID: "781a9e72-e05b-4bb1-9a3d-a6b960d37809"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:44:22.800421 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.800395    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "781a9e72-e05b-4bb1-9a3d-a6b960d37809" (UID: "781a9e72-e05b-4bb1-9a3d-a6b960d37809"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:44:22.800534 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.800407    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/781a9e72-e05b-4bb1-9a3d-a6b960d37809-kube-api-access-l5h6k" (OuterVolumeSpecName: "kube-api-access-l5h6k") pod "781a9e72-e05b-4bb1-9a3d-a6b960d37809" (UID: "781a9e72-e05b-4bb1-9a3d-a6b960d37809"). InnerVolumeSpecName "kube-api-access-l5h6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:44:22.800534 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.800400    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-dshm" (OuterVolumeSpecName: "dshm") pod "781a9e72-e05b-4bb1-9a3d-a6b960d37809" (UID: "781a9e72-e05b-4bb1-9a3d-a6b960d37809"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:44:22.898927 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.898888    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tls-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:22.898927 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.898917    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-home\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:22.898927 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.898926    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5h6k\" (UniqueName: \"kubernetes.io/projected/781a9e72-e05b-4bb1-9a3d-a6b960d37809-kube-api-access-l5h6k\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:22.898927 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.898935    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-dshm\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:22.899222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.898943    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-tmp-dir\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:22.899222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:22.898951    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/781a9e72-e05b-4bb1-9a3d-a6b960d37809-model-cache\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:44:23.177586 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:23.177548    2583 generic.go:356] "Generic (PLEG): container finished" podID="781a9e72-e05b-4bb1-9a3d-a6b960d37809" containerID="5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66" exitCode=0
Sep 11 19:44:23.177799 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:23.177592    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"
Sep 11 19:44:23.177799 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:23.177637    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt" event={"ID":"781a9e72-e05b-4bb1-9a3d-a6b960d37809","Type":"ContainerDied","Data":"5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66"}
Sep 11 19:44:23.177799 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:23.177683    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt" event={"ID":"781a9e72-e05b-4bb1-9a3d-a6b960d37809","Type":"ContainerDied","Data":"0623516eb6f13e65bf5654fe8124f0870a5c46c644db86d028923b867415ef58"}
Sep 11 19:44:23.177799 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:23.177700    2583 scope.go:117] "RemoveContainer" containerID="5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66"
Sep 11 19:44:23.186740 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:23.186721    2583 scope.go:117] "RemoveContainer" containerID="5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66"
Sep 11 19:44:23.187023 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:44:23.187005    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66\": container with ID starting with 5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66 not found: ID does not exist" containerID="5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66"
Sep 11 19:44:23.187072 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:23.187032    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66"} err="failed to get container status \"5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66\": rpc error: code = NotFound desc = could not find container \"5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66\": container with ID starting with 5913b3a07b5b8d985c07dc310c8fdcafc60b92145c0c2d98a196ab6f0daf9c66 not found: ID does not exist"
Sep 11 19:44:23.198359 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:23.198339    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"]
Sep 11 19:44:23.202572 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:23.202549    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/auth-test-model-abcdff-kserve-964854fbc-rh6vt"]
Sep 11 19:44:24.546900 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:44:24.546861    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="781a9e72-e05b-4bb1-9a3d-a6b960d37809" path="/var/lib/kubelet/pods/781a9e72-e05b-4bb1-9a3d-a6b960d37809/volumes"
Sep 11 19:45:01.248130 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:01.248092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:45:17.686016 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:17.685984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:45:17.786810 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:17.786775    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:45:20.209497 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.209458    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"]
Sep 11 19:45:20.210067 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210040    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5203fea-7020-4559-b071-36df336e300b" containerName="maas-api"
Sep 11 19:45:20.210067 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210064    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5203fea-7020-4559-b071-36df336e300b" containerName="maas-api"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210080    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3" containerName="maas-api"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210089    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3" containerName="maas-api"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210105    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44570393-a4ea-4b74-8575-1fd612eb4cd9" containerName="maas-api"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210111    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="44570393-a4ea-4b74-8575-1fd612eb4cd9" containerName="maas-api"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210120    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb451d72-d2c8-4796-979e-4a00420ab008" containerName="main"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210125    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb451d72-d2c8-4796-979e-4a00420ab008" containerName="main"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210132    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="781a9e72-e05b-4bb1-9a3d-a6b960d37809" containerName="main"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210137    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="781a9e72-e05b-4bb1-9a3d-a6b960d37809" containerName="main"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210145    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd0b9d1-a040-4b61-9bac-30413990044e" containerName="payload-pre-processing"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210151    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd0b9d1-a040-4b61-9bac-30413990044e" containerName="payload-pre-processing"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210160    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c83f28ea-8770-4fd1-9708-9dea39965fe3" containerName="main"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210167    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c83f28ea-8770-4fd1-9708-9dea39965fe3" containerName="main"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210176    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01b47d19-c133-4339-820e-56a8521a4994" containerName="payload-processing"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210186    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="01b47d19-c133-4339-820e-56a8521a4994" containerName="payload-processing"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210195    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57932cd5-9b5f-48e4-b747-572df168c661" containerName="main"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210200    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="57932cd5-9b5f-48e4-b747-572df168c661" containerName="main"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210281    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5203fea-7020-4559-b071-36df336e300b" containerName="maas-api"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210289    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb451d72-d2c8-4796-979e-4a00420ab008" containerName="main"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210296    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="44570393-a4ea-4b74-8575-1fd612eb4cd9" containerName="maas-api"
Sep 11 19:45:20.210287 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210302    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c83f28ea-8770-4fd1-9708-9dea39965fe3" containerName="main"
Sep 11 19:45:20.210938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210315    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e4d73a5-29c3-4bbe-a0ae-e4e381b4a4c3" containerName="maas-api"
Sep 11 19:45:20.210938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210324    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd0b9d1-a040-4b61-9bac-30413990044e" containerName="payload-pre-processing"
Sep 11 19:45:20.210938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210331    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="01b47d19-c133-4339-820e-56a8521a4994" containerName="payload-processing"
Sep 11 19:45:20.210938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210339    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="781a9e72-e05b-4bb1-9a3d-a6b960d37809" containerName="main"
Sep 11 19:45:20.210938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.210350    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="57932cd5-9b5f-48e4-b747-572df168c661" containerName="main"
Sep 11 19:45:20.214812 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.214791    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.217809 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.217785    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-abcdff\"/\"kube-root-ca.crt\""
Sep 11 19:45:20.218594 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.218571    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-abcdff\"/\"default-dockercfg-lv6pr\""
Sep 11 19:45:20.218727 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.218571    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-abcdff\"/\"openshift-service-ca.crt\""
Sep 11 19:45:20.218727 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.218632    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-abcdff\"/\"sub-test-model-abcdff-kserve-self-signed-certs\""
Sep 11 19:45:20.222929 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.222804    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"]
Sep 11 19:45:20.238093 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.238069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:45:20.317475 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.317433    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvwr6\" (UniqueName: \"kubernetes.io/projected/5c026aba-5c22-40d4-ac6d-6db13ff31571-kube-api-access-bvwr6\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.317665 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.317502    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-tmp-dir\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.317665 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.317529    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c026aba-5c22-40d4-ac6d-6db13ff31571-tls-certs\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.317665 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.317576    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-dshm\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.317665 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.317640    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-model-cache\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.317861 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.317693    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-home\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.418612 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.418574    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-tmp-dir\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.418828 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.418621    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c026aba-5c22-40d4-ac6d-6db13ff31571-tls-certs\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.418828 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.418680    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-dshm\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.418828 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.418702    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-model-cache\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.418828 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.418725    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-home\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.418828 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.418802    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvwr6\" (UniqueName: \"kubernetes.io/projected/5c026aba-5c22-40d4-ac6d-6db13ff31571-kube-api-access-bvwr6\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.419098 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.419040    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-tmp-dir\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.419098 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.419060    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-model-cache\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.419203 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.419143    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-home\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.421204 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.421181    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-dshm\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.421368 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.421352    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c026aba-5c22-40d4-ac6d-6db13ff31571-tls-certs\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.426898 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.426878    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvwr6\" (UniqueName: \"kubernetes.io/projected/5c026aba-5c22-40d4-ac6d-6db13ff31571-kube-api-access-bvwr6\") pod \"sub-test-model-abcdff-kserve-c8d6c694c-4qht7\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") " pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.526416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.526328    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:20.814107 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:20.814034    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"]
Sep 11 19:45:20.817628 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:45:20.817605    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c026aba_5c22_40d4_ac6d_6db13ff31571.slice/crio-74fa2becdd8bda6fe8ef7347d072e83ab4efd7257e4cd90d82b3f4f666079fda WatchSource:0}: Error finding container 74fa2becdd8bda6fe8ef7347d072e83ab4efd7257e4cd90d82b3f4f666079fda: Status 404 returned error can't find the container with id 74fa2becdd8bda6fe8ef7347d072e83ab4efd7257e4cd90d82b3f4f666079fda
Sep 11 19:45:21.455057 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:21.455017    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7" event={"ID":"5c026aba-5c22-40d4-ac6d-6db13ff31571","Type":"ContainerStarted","Data":"f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9"}
Sep 11 19:45:21.455479 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:21.455063    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7" event={"ID":"5c026aba-5c22-40d4-ac6d-6db13ff31571","Type":"ContainerStarted","Data":"74fa2becdd8bda6fe8ef7347d072e83ab4efd7257e4cd90d82b3f4f666079fda"}
Sep 11 19:45:21.473345 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:21.473295    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7" podStartSLOduration=1.4732802010000001 podStartE2EDuration="1.473280201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:45:21.471711264 +0000 UTC m=+1625.400336175" watchObservedRunningTime="2026-09-11 19:45:21.473280201 +0000 UTC m=+1625.401905099"
Sep 11 19:45:24.225346 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:24.225314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:45:24.325174 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:24.325141    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:45:30.527509 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:30.527456    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:30.528039 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:30.527529    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:30.546107 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:30.546070    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:31.516194 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:31.516161    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:45:37.350076 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:37.350045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:45:51.654023 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:51.653985    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7801-27fd2f"]
Sep 11 19:45:51.657383 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:51.657366    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-27fd2f"
Sep 11 19:45:51.659637 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:51.659618    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-v69wf\""
Sep 11 19:45:51.663284 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:51.663262    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-27fd2f"]
Sep 11 19:45:51.727837 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:51.727796    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v85vt\" (UniqueName: \"kubernetes.io/projected/0f882b5f-fe0a-42af-8b74-c910c1162fda-kube-api-access-v85vt\") pod \"test-curl-7801-27fd2f\" (UID: \"0f882b5f-fe0a-42af-8b74-c910c1162fda\") " pod="opendatahub/test-curl-7801-27fd2f"
Sep 11 19:45:51.829147 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:51.829111    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v85vt\" (UniqueName: \"kubernetes.io/projected/0f882b5f-fe0a-42af-8b74-c910c1162fda-kube-api-access-v85vt\") pod \"test-curl-7801-27fd2f\" (UID: \"0f882b5f-fe0a-42af-8b74-c910c1162fda\") " pod="opendatahub/test-curl-7801-27fd2f"
Sep 11 19:45:51.837507 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:51.837479    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v85vt\" (UniqueName: \"kubernetes.io/projected/0f882b5f-fe0a-42af-8b74-c910c1162fda-kube-api-access-v85vt\") pod \"test-curl-7801-27fd2f\" (UID: \"0f882b5f-fe0a-42af-8b74-c910c1162fda\") " pod="opendatahub/test-curl-7801-27fd2f"
Sep 11 19:45:51.968623 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:51.968538    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-27fd2f"
Sep 11 19:45:52.236399 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:52.236376    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-27fd2f"]
Sep 11 19:45:52.238797 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:45:52.238771    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f882b5f_fe0a_42af_8b74_c910c1162fda.slice/crio-18d23f060e46bb021c522c42be3f3bb012e9d2926e321796d12bbb072d07f85f WatchSource:0}: Error finding container 18d23f060e46bb021c522c42be3f3bb012e9d2926e321796d12bbb072d07f85f: Status 404 returned error can't find the container with id 18d23f060e46bb021c522c42be3f3bb012e9d2926e321796d12bbb072d07f85f
Sep 11 19:45:52.602137 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:52.602105    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-27fd2f" event={"ID":"0f882b5f-fe0a-42af-8b74-c910c1162fda","Type":"ContainerStarted","Data":"a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae"}
Sep 11 19:45:52.602329 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:52.602150    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-27fd2f" event={"ID":"0f882b5f-fe0a-42af-8b74-c910c1162fda","Type":"ContainerStarted","Data":"18d23f060e46bb021c522c42be3f3bb012e9d2926e321796d12bbb072d07f85f"}
Sep 11 19:45:52.617034 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:52.616988    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7801-27fd2f" podStartSLOduration=1.616971719 podStartE2EDuration="1.616971719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:45:52.615232295 +0000 UTC m=+1656.543857193" watchObservedRunningTime="2026-09-11 19:45:52.616971719 +0000 UTC m=+1656.545596617"
Sep 11 19:45:53.174094 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.174059    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7801-27fd2f"]
Sep 11 19:45:53.179945 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.179917    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7801-27fd2f"]
Sep 11 19:45:53.298254 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.298164    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7801-123864"]
Sep 11 19:45:53.298571 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.298555    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f882b5f-fe0a-42af-8b74-c910c1162fda" containerName="test-curl-7801-27fd2f"
Sep 11 19:45:53.298571 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.298572    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f882b5f-fe0a-42af-8b74-c910c1162fda" containerName="test-curl-7801-27fd2f"
Sep 11 19:45:53.298652 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.298638    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f882b5f-fe0a-42af-8b74-c910c1162fda" containerName="test-curl-7801-27fd2f"
Sep 11 19:45:53.301612 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.301596    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-123864"
Sep 11 19:45:53.306420 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.306396    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-123864"]
Sep 11 19:45:53.445673 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.445636    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gbq5\" (UniqueName: \"kubernetes.io/projected/f3d93615-df93-4989-b74a-ad6ccfd93b65-kube-api-access-8gbq5\") pod \"test-curl-7801-123864\" (UID: \"f3d93615-df93-4989-b74a-ad6ccfd93b65\") " pod="opendatahub/test-curl-7801-123864"
Sep 11 19:45:53.546745 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.546702    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gbq5\" (UniqueName: \"kubernetes.io/projected/f3d93615-df93-4989-b74a-ad6ccfd93b65-kube-api-access-8gbq5\") pod \"test-curl-7801-123864\" (UID: \"f3d93615-df93-4989-b74a-ad6ccfd93b65\") " pod="opendatahub/test-curl-7801-123864"
Sep 11 19:45:53.555770 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.555708    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gbq5\" (UniqueName: \"kubernetes.io/projected/f3d93615-df93-4989-b74a-ad6ccfd93b65-kube-api-access-8gbq5\") pod \"test-curl-7801-123864\" (UID: \"f3d93615-df93-4989-b74a-ad6ccfd93b65\") " pod="opendatahub/test-curl-7801-123864"
Sep 11 19:45:53.606490 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:45:53.606452    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7801-27fd2f" container="test-curl-7801-27fd2f"
Sep 11 19:45:53.612502 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.612477    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-123864"
Sep 11 19:45:53.880702 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:53.880674    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-123864"]
Sep 11 19:45:53.882975 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:45:53.882945    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3d93615_df93_4989_b74a_ad6ccfd93b65.slice/crio-ba0806738b768ead09267f123b6d19e8c72ac328265306e8c1141453f050f46b WatchSource:0}: Error finding container ba0806738b768ead09267f123b6d19e8c72ac328265306e8c1141453f050f46b: Status 404 returned error can't find the container with id ba0806738b768ead09267f123b6d19e8c72ac328265306e8c1141453f050f46b
Sep 11 19:45:54.612736 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:54.612698    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-123864" event={"ID":"f3d93615-df93-4989-b74a-ad6ccfd93b65","Type":"ContainerStarted","Data":"7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135"}
Sep 11 19:45:54.612736 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:54.612742    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-123864" event={"ID":"f3d93615-df93-4989-b74a-ad6ccfd93b65","Type":"ContainerStarted","Data":"ba0806738b768ead09267f123b6d19e8c72ac328265306e8c1141453f050f46b"}
Sep 11 19:45:54.613261 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:54.612756    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7801-27fd2f" podUID="0f882b5f-fe0a-42af-8b74-c910c1162fda" containerName="test-curl-7801-27fd2f" containerID="cri-o://a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae" gracePeriod=2
Sep 11 19:45:54.627389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:54.627343    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7801-123864" podStartSLOduration=1.627322731 podStartE2EDuration="1.627322731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:45:54.625753511 +0000 UTC m=+1658.554378408" watchObservedRunningTime="2026-09-11 19:45:54.627322731 +0000 UTC m=+1658.555947628"
Sep 11 19:45:55.196383 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:55.196349    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7801-123864"]
Sep 11 19:45:55.200804 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:55.200775    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7801-123864"]
Sep 11 19:45:55.617326 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:45:55.617294    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7801-123864" container="test-curl-7801-123864"
Sep 11 19:45:56.621999 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:56.621964    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7801-123864" podUID="f3d93615-df93-4989-b74a-ad6ccfd93b65" containerName="test-curl-7801-123864" containerID="cri-o://7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135" gracePeriod=2
Sep 11 19:45:56.763381 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:56.763350    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-27fd2f"
Sep 11 19:45:56.877636 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:56.877535    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v85vt\" (UniqueName: \"kubernetes.io/projected/0f882b5f-fe0a-42af-8b74-c910c1162fda-kube-api-access-v85vt\") pod \"0f882b5f-fe0a-42af-8b74-c910c1162fda\" (UID: \"0f882b5f-fe0a-42af-8b74-c910c1162fda\") "
Sep 11 19:45:56.879754 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:56.879729    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f882b5f-fe0a-42af-8b74-c910c1162fda-kube-api-access-v85vt" (OuterVolumeSpecName: "kube-api-access-v85vt") pod "0f882b5f-fe0a-42af-8b74-c910c1162fda" (UID: "0f882b5f-fe0a-42af-8b74-c910c1162fda"). InnerVolumeSpecName "kube-api-access-v85vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:45:56.978710 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:56.978670    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v85vt\" (UniqueName: \"kubernetes.io/projected/0f882b5f-fe0a-42af-8b74-c910c1162fda-kube-api-access-v85vt\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:45:57.206288 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:57.206193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:45:57.628826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:57.628788    2583 generic.go:356] "Generic (PLEG): container finished" podID="0f882b5f-fe0a-42af-8b74-c910c1162fda" containerID="a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae" exitCode=137
Sep 11 19:45:57.629318 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:57.628833    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-27fd2f"
Sep 11 19:45:57.629318 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:57.628906    2583 scope.go:117] "RemoveContainer" containerID="a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae"
Sep 11 19:45:57.637845 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:57.637826    2583 scope.go:117] "RemoveContainer" containerID="a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae"
Sep 11 19:45:57.638149 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:45:57.638130    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae\": container with ID starting with a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae not found: ID does not exist" containerID="a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae"
Sep 11 19:45:57.638203 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:57.638160    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae"} err="failed to get container status \"a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae\": rpc error: code = NotFound desc = could not find container \"a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae\": container with ID starting with a1241ad182b0c3ebaf531ee0fbe261d2fd5cf32cffea0169a887154a614be3ae not found: ID does not exist"
Sep 11 19:45:57.638452 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:45:57.638431    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7801-27fd2f\" is forbidden: User \"system:node:ip-10-0-135-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-147.ec2.internal' and this object" podUID="0f882b5f-fe0a-42af-8b74-c910c1162fda" pod="opendatahub/test-curl-7801-27fd2f"
Sep 11 19:45:58.120144 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:58.120109    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"]
Sep 11 19:45:58.122401 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:45:58.122368    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7801-27fd2f\" is forbidden: User \"system:node:ip-10-0-135-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-147.ec2.internal' and this object" podUID="0f882b5f-fe0a-42af-8b74-c910c1162fda" pod="opendatahub/test-curl-7801-27fd2f"
Sep 11 19:45:58.140372 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:58.140343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:45:58.546659 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:58.546559    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f882b5f-fe0a-42af-8b74-c910c1162fda" path="/var/lib/kubelet/pods/0f882b5f-fe0a-42af-8b74-c910c1162fda/volumes"
Sep 11 19:45:58.796766 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:58.796706    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-123864"
Sep 11 19:45:58.895751 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:58.895718    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gbq5\" (UniqueName: \"kubernetes.io/projected/f3d93615-df93-4989-b74a-ad6ccfd93b65-kube-api-access-8gbq5\") pod \"f3d93615-df93-4989-b74a-ad6ccfd93b65\" (UID: \"f3d93615-df93-4989-b74a-ad6ccfd93b65\") "
Sep 11 19:45:58.897881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:58.897858    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d93615-df93-4989-b74a-ad6ccfd93b65-kube-api-access-8gbq5" (OuterVolumeSpecName: "kube-api-access-8gbq5") pod "f3d93615-df93-4989-b74a-ad6ccfd93b65" (UID: "f3d93615-df93-4989-b74a-ad6ccfd93b65"). InnerVolumeSpecName "kube-api-access-8gbq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:45:58.996752 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:58.996715    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gbq5\" (UniqueName: \"kubernetes.io/projected/f3d93615-df93-4989-b74a-ad6ccfd93b65-kube-api-access-8gbq5\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:45:59.641537 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:59.641493    2583 generic.go:356] "Generic (PLEG): container finished" podID="f3d93615-df93-4989-b74a-ad6ccfd93b65" containerID="7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135" exitCode=137
Sep 11 19:45:59.641756 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:59.641550    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-123864"
Sep 11 19:45:59.641756 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:59.641623    2583 scope.go:117] "RemoveContainer" containerID="7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135"
Sep 11 19:45:59.651349 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:59.651325    2583 scope.go:117] "RemoveContainer" containerID="7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135"
Sep 11 19:45:59.651637 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:45:59.651619    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135\": container with ID starting with 7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135 not found: ID does not exist" containerID="7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135"
Sep 11 19:45:59.651712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:45:59.651651    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135"} err="failed to get container status \"7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135\": rpc error: code = NotFound desc = could not find container \"7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135\": container with ID starting with 7a83479109670e2dceb8a09cfa57073f51cc7228d8810c28d9eca5bacd2fe135 not found: ID does not exist"
Sep 11 19:45:59.652152 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:45:59.652129    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7801-123864\" is forbidden: User \"system:node:ip-10-0-135-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-147.ec2.internal' and this object" podUID="f3d93615-df93-4989-b74a-ad6ccfd93b65" pod="opendatahub/test-curl-7801-123864"
Sep 11 19:46:00.546681 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:00.546644    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3d93615-df93-4989-b74a-ad6ccfd93b65" path="/var/lib/kubelet/pods/f3d93615-df93-4989-b74a-ad6ccfd93b65/volumes"
Sep 11 19:46:03.804130 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.804091    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"]
Sep 11 19:46:03.804778 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.804750    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d93615-df93-4989-b74a-ad6ccfd93b65" containerName="test-curl-7801-123864"
Sep 11 19:46:03.804778 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.804782    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d93615-df93-4989-b74a-ad6ccfd93b65" containerName="test-curl-7801-123864"
Sep 11 19:46:03.804950 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.804885    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d93615-df93-4989-b74a-ad6ccfd93b65" containerName="test-curl-7801-123864"
Sep 11 19:46:03.809931 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.809910    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:03.812458 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.812435    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-19859a\"/\"kube-root-ca.crt\""
Sep 11 19:46:03.813427 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.813277    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-19859a\"/\"default-dockercfg-l7jwf\""
Sep 11 19:46:03.813427 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.813299    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-19859a\"/\"rate-test-model-19859a-kserve-self-signed-certs\""
Sep 11 19:46:03.813427 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.813314    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-19859a\"/\"openshift-service-ca.crt\""
Sep 11 19:46:03.817484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.817431    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"]
Sep 11 19:46:03.842880 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.842846    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:46:03.944629 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.944597    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-model-cache\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:03.944816 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.944635    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-home\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:03.944816 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.944665    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tls-certs\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:03.944816 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.944751    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tmp-dir\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:03.944936 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.944836    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-dshm\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:03.944936 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:03.944867    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2vr5\" (UniqueName: \"kubernetes.io/projected/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-kube-api-access-p2vr5\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.046347 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.046304    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-dshm\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.046528 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.046365    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2vr5\" (UniqueName: \"kubernetes.io/projected/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-kube-api-access-p2vr5\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.046528 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.046405    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-model-cache\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.046528 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.046433    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-home\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.046528 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.046471    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tls-certs\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.046528 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.046508    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tmp-dir\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.046823 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.046804    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-model-cache\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.046885 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.046812    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-home\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.046885 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.046860    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tmp-dir\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.048802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.048781    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-dshm\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.048962 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.048948    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tls-certs\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.054846 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.054785    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2vr5\" (UniqueName: \"kubernetes.io/projected/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-kube-api-access-p2vr5\") pod \"rate-test-model-19859a-kserve-584dc57655-mgjqz\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") " pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.121808 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.121766    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:04.394392 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.394367    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"]
Sep 11 19:46:04.396222 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:46:04.396198    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f9e1503_9c09_49dd_9b95_310bc9c6c81b.slice/crio-a7608e0c5d024de3d2ecc120d376abf13cc80e0ef1ed8fe1fbf975169507613d WatchSource:0}: Error finding container a7608e0c5d024de3d2ecc120d376abf13cc80e0ef1ed8fe1fbf975169507613d: Status 404 returned error can't find the container with id a7608e0c5d024de3d2ecc120d376abf13cc80e0ef1ed8fe1fbf975169507613d
Sep 11 19:46:04.666510 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.666422    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz" event={"ID":"2f9e1503-9c09-49dd-9b95-310bc9c6c81b","Type":"ContainerStarted","Data":"40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2"}
Sep 11 19:46:04.666510 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.666460    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz" event={"ID":"2f9e1503-9c09-49dd-9b95-310bc9c6c81b","Type":"ContainerStarted","Data":"a7608e0c5d024de3d2ecc120d376abf13cc80e0ef1ed8fe1fbf975169507613d"}
Sep 11 19:46:04.682382 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:04.682330    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz" podStartSLOduration=1.68230952 podStartE2EDuration="1.68230952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:46:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:46:04.681095722 +0000 UTC m=+1668.609720620" watchObservedRunningTime="2026-09-11 19:46:04.68230952 +0000 UTC m=+1668.610934419"
Sep 11 19:46:13.139801 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.139762    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7" podUID="5c026aba-5c22-40d4-ac6d-6db13ff31571" containerName="main" containerID="cri-o://f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9" gracePeriod=45
Sep 11 19:46:13.390337 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.390274    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:46:13.430781 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.430751    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-model-cache\") pod \"5c026aba-5c22-40d4-ac6d-6db13ff31571\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") "
Sep 11 19:46:13.430946 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.430795    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-home\") pod \"5c026aba-5c22-40d4-ac6d-6db13ff31571\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") "
Sep 11 19:46:13.430946 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.430834    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-dshm\") pod \"5c026aba-5c22-40d4-ac6d-6db13ff31571\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") "
Sep 11 19:46:13.430946 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.430875    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-tmp-dir\") pod \"5c026aba-5c22-40d4-ac6d-6db13ff31571\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") "
Sep 11 19:46:13.431085 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.430969    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvwr6\" (UniqueName: \"kubernetes.io/projected/5c026aba-5c22-40d4-ac6d-6db13ff31571-kube-api-access-bvwr6\") pod \"5c026aba-5c22-40d4-ac6d-6db13ff31571\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") "
Sep 11 19:46:13.431085 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.430999    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c026aba-5c22-40d4-ac6d-6db13ff31571-tls-certs\") pod \"5c026aba-5c22-40d4-ac6d-6db13ff31571\" (UID: \"5c026aba-5c22-40d4-ac6d-6db13ff31571\") "
Sep 11 19:46:13.431085 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.431020    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-model-cache" (OuterVolumeSpecName: "model-cache") pod "5c026aba-5c22-40d4-ac6d-6db13ff31571" (UID: "5c026aba-5c22-40d4-ac6d-6db13ff31571"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:46:13.431261 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.431087    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-home" (OuterVolumeSpecName: "home") pod "5c026aba-5c22-40d4-ac6d-6db13ff31571" (UID: "5c026aba-5c22-40d4-ac6d-6db13ff31571"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:46:13.431261 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.431124    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5c026aba-5c22-40d4-ac6d-6db13ff31571" (UID: "5c026aba-5c22-40d4-ac6d-6db13ff31571"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:46:13.431374 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.431312    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-model-cache\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:46:13.431374 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.431325    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-home\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:46:13.431374 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.431333    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-tmp-dir\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:46:13.433048 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.433021    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-dshm" (OuterVolumeSpecName: "dshm") pod "5c026aba-5c22-40d4-ac6d-6db13ff31571" (UID: "5c026aba-5c22-40d4-ac6d-6db13ff31571"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:46:13.433161 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.433102    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c026aba-5c22-40d4-ac6d-6db13ff31571-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5c026aba-5c22-40d4-ac6d-6db13ff31571" (UID: "5c026aba-5c22-40d4-ac6d-6db13ff31571"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:46:13.433220 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.433203    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c026aba-5c22-40d4-ac6d-6db13ff31571-kube-api-access-bvwr6" (OuterVolumeSpecName: "kube-api-access-bvwr6") pod "5c026aba-5c22-40d4-ac6d-6db13ff31571" (UID: "5c026aba-5c22-40d4-ac6d-6db13ff31571"). InnerVolumeSpecName "kube-api-access-bvwr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:46:13.532196 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.532160    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c026aba-5c22-40d4-ac6d-6db13ff31571-dshm\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:46:13.532196 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.532189    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvwr6\" (UniqueName: \"kubernetes.io/projected/5c026aba-5c22-40d4-ac6d-6db13ff31571-kube-api-access-bvwr6\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:46:13.532196 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.532200    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c026aba-5c22-40d4-ac6d-6db13ff31571-tls-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:46:13.710430 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.710340    2583 generic.go:356] "Generic (PLEG): container finished" podID="5c026aba-5c22-40d4-ac6d-6db13ff31571" containerID="f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9" exitCode=0
Sep 11 19:46:13.710430 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.710390    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"
Sep 11 19:46:13.710430 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.710409    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7" event={"ID":"5c026aba-5c22-40d4-ac6d-6db13ff31571","Type":"ContainerDied","Data":"f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9"}
Sep 11 19:46:13.710697 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.710443    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7" event={"ID":"5c026aba-5c22-40d4-ac6d-6db13ff31571","Type":"ContainerDied","Data":"74fa2becdd8bda6fe8ef7347d072e83ab4efd7257e4cd90d82b3f4f666079fda"}
Sep 11 19:46:13.710697 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.710459    2583 scope.go:117] "RemoveContainer" containerID="f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9"
Sep 11 19:46:13.725341 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.725317    2583 scope.go:117] "RemoveContainer" containerID="f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9"
Sep 11 19:46:13.725673 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:46:13.725653    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9\": container with ID starting with f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9 not found: ID does not exist" containerID="f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9"
Sep 11 19:46:13.725760 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.725680    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9"} err="failed to get container status \"f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9\": rpc error: code = NotFound desc = could not find container \"f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9\": container with ID starting with f24663fede7b7bef9ada9a0d3d6792aed0fa47e3a46bcf2c4b7bf1a7e78df7f9 not found: ID does not exist"
Sep 11 19:46:13.734812 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.734783    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"]
Sep 11 19:46:13.738178 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:13.738152    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-abcdff/sub-test-model-abcdff-kserve-c8d6c694c-4qht7"]
Sep 11 19:46:14.122314 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:14.122276    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:14.122493 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:14.122342    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:14.135074 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:14.135047    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:14.547177 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:14.547100    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c026aba-5c22-40d4-ac6d-6db13ff31571" path="/var/lib/kubelet/pods/5c026aba-5c22-40d4-ac6d-6db13ff31571/volumes"
Sep 11 19:46:14.728156 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:14.728122    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:46:25.741473 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:46:25.741437    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:04.731188 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:04.731144    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"]
Sep 11 19:47:04.750217 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:04.750186    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:05.813876 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:05.813844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:06.230847 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:06.230815    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:06.607210 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:06.607180    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:06.750040 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:06.750001    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:08.740525 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:08.740490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:09.443832 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:09.443797    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:09.626467 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:09.626431    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:09.703884 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:09.703802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:10.439391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:10.439352    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:11.254964 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.254922    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"]
Sep 11 19:47:11.256203 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.256171    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c026aba-5c22-40d4-ac6d-6db13ff31571" containerName="main"
Sep 11 19:47:11.256203 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.256203    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c026aba-5c22-40d4-ac6d-6db13ff31571" containerName="main"
Sep 11 19:47:11.256421 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.256407    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c026aba-5c22-40d4-ac6d-6db13ff31571" containerName="main"
Sep 11 19:47:11.261426 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.261400    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.264005 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.263980    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-c0a42d-serving-cert\""
Sep 11 19:47:11.281579 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.281551    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"]
Sep 11 19:47:11.301977 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.301949    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:11.370437 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.370392    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd0288d1-15eb-4c2e-acff-b821666d171c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.370613 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.370504    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.370666 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.370621    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szs6q\" (UniqueName: \"kubernetes.io/projected/fd0288d1-15eb-4c2e-acff-b821666d171c-kube-api-access-szs6q\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.370720 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.370663    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-metrics-certs\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.411124 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.411095    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:11.471636 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.471599    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.472033 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.471708    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szs6q\" (UniqueName: \"kubernetes.io/projected/fd0288d1-15eb-4c2e-acff-b821666d171c-kube-api-access-szs6q\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.472033 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.471724    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-metrics-certs\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.472033 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:47:11.471742    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-serving-cert: secret "maas-api-e2e-ipp-a-c0a42d-serving-cert" not found
Sep 11 19:47:11.472033 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.471759    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd0288d1-15eb-4c2e-acff-b821666d171c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.472033 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:47:11.471832    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-maas-api-tls podName:fd0288d1-15eb-4c2e-acff-b821666d171c nodeName:}" failed. No retries permitted until 2026-09-11 19:47:11.971807054 +0000 UTC m=+1735.900431948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-maas-api-tls") pod "maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf" (UID: "fd0288d1-15eb-4c2e-acff-b821666d171c") : secret "maas-api-e2e-ipp-a-c0a42d-serving-cert" not found
Sep 11 19:47:11.472394 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.472375    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd0288d1-15eb-4c2e-acff-b821666d171c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.474356 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.474339    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-metrics-certs\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.480556 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.480532    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szs6q\" (UniqueName: \"kubernetes.io/projected/fd0288d1-15eb-4c2e-acff-b821666d171c-kube-api-access-szs6q\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.672218 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.672188    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:11.923092 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.923005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:11.976209 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.976182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:11.978669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:11.978649    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:12.014555 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:12.014523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:12.172813 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:12.172778    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:12.450799 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:12.450772    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"]
Sep 11 19:47:12.452850 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:47:12.452825    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0288d1_15eb_4c2e_acff_b821666d171c.slice/crio-c5f851004ba4d15158b0de357b7828072bc41a705d7e0f7e6273b31e05105ad4 WatchSource:0}: Error finding container c5f851004ba4d15158b0de357b7828072bc41a705d7e0f7e6273b31e05105ad4: Status 404 returned error can't find the container with id c5f851004ba4d15158b0de357b7828072bc41a705d7e0f7e6273b31e05105ad4
Sep 11 19:47:13.009659 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:13.009619    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf" event={"ID":"fd0288d1-15eb-4c2e-acff-b821666d171c","Type":"ContainerStarted","Data":"c5f851004ba4d15158b0de357b7828072bc41a705d7e0f7e6273b31e05105ad4"}
Sep 11 19:47:14.016503 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:14.016469    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf" event={"ID":"fd0288d1-15eb-4c2e-acff-b821666d171c","Type":"ContainerStarted","Data":"d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64"}
Sep 11 19:47:14.016891 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:14.016580    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:14.035997 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:14.035951    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf" podStartSLOduration=2.109561272 podStartE2EDuration="3.035935566s" totalImagesPullingTime="926.374294ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:47:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:47:14.034605598 +0000 UTC m=+1737.963230495" watchObservedRunningTime="2026-09-11 19:47:14.035935566 +0000 UTC m=+1737.964560463"
Sep 11 19:47:19.751734 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:19.751662    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz" podUID="2f9e1503-9c09-49dd-9b95-310bc9c6c81b" containerName="main" containerID="cri-o://40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2" gracePeriod=45
Sep 11 19:47:19.996479 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:19.996456    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:47:20.026995 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.026920    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:47:20.048626 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.048595    2583 generic.go:356] "Generic (PLEG): container finished" podID="2f9e1503-9c09-49dd-9b95-310bc9c6c81b" containerID="40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2" exitCode=0
Sep 11 19:47:20.048802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.048632    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz" event={"ID":"2f9e1503-9c09-49dd-9b95-310bc9c6c81b","Type":"ContainerDied","Data":"40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2"}
Sep 11 19:47:20.048802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.048637    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"
Sep 11 19:47:20.048802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.048665    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz" event={"ID":"2f9e1503-9c09-49dd-9b95-310bc9c6c81b","Type":"ContainerDied","Data":"a7608e0c5d024de3d2ecc120d376abf13cc80e0ef1ed8fe1fbf975169507613d"}
Sep 11 19:47:20.048802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.048688    2583 scope.go:117] "RemoveContainer" containerID="40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2"
Sep 11 19:47:20.061645 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.059131    2583 scope.go:117] "RemoveContainer" containerID="40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2"
Sep 11 19:47:20.061926 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:47:20.061897    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2\": container with ID starting with 40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2 not found: ID does not exist" containerID="40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2"
Sep 11 19:47:20.062024 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.061939    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2"} err="failed to get container status \"40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2\": rpc error: code = NotFound desc = could not find container \"40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2\": container with ID starting with 40226a886f5f8373468f341e2957a0ffcb25006e33a7dabe78ef08a9b44f30d2 not found: ID does not exist"
Sep 11 19:47:20.155212 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.155182    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-dshm\") pod \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") "
Sep 11 19:47:20.155389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.155251    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tls-certs\") pod \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") "
Sep 11 19:47:20.155389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.155299    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tmp-dir\") pod \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") "
Sep 11 19:47:20.155389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.155322    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-home\") pod \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") "
Sep 11 19:47:20.155389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.155373    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2vr5\" (UniqueName: \"kubernetes.io/projected/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-kube-api-access-p2vr5\") pod \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") "
Sep 11 19:47:20.155568 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.155407    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-model-cache\") pod \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\" (UID: \"2f9e1503-9c09-49dd-9b95-310bc9c6c81b\") "
Sep 11 19:47:20.155627 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.155557    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2f9e1503-9c09-49dd-9b95-310bc9c6c81b" (UID: "2f9e1503-9c09-49dd-9b95-310bc9c6c81b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:47:20.155687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.155616    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-home" (OuterVolumeSpecName: "home") pod "2f9e1503-9c09-49dd-9b95-310bc9c6c81b" (UID: "2f9e1503-9c09-49dd-9b95-310bc9c6c81b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:47:20.155811 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.155759    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-model-cache" (OuterVolumeSpecName: "model-cache") pod "2f9e1503-9c09-49dd-9b95-310bc9c6c81b" (UID: "2f9e1503-9c09-49dd-9b95-310bc9c6c81b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:47:20.156024 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.156009    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tmp-dir\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:47:20.156104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.156031    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-home\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:47:20.157437 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.157409    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-dshm" (OuterVolumeSpecName: "dshm") pod "2f9e1503-9c09-49dd-9b95-310bc9c6c81b" (UID: "2f9e1503-9c09-49dd-9b95-310bc9c6c81b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:47:20.157581 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.157565    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2f9e1503-9c09-49dd-9b95-310bc9c6c81b" (UID: "2f9e1503-9c09-49dd-9b95-310bc9c6c81b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:47:20.157581 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.157568    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-kube-api-access-p2vr5" (OuterVolumeSpecName: "kube-api-access-p2vr5") pod "2f9e1503-9c09-49dd-9b95-310bc9c6c81b" (UID: "2f9e1503-9c09-49dd-9b95-310bc9c6c81b"). InnerVolumeSpecName "kube-api-access-p2vr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:47:20.256898 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.256866    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-tls-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:47:20.256898 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.256897    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2vr5\" (UniqueName: \"kubernetes.io/projected/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-kube-api-access-p2vr5\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:47:20.256898 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.256911    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-model-cache\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:47:20.257159 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.256923    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f9e1503-9c09-49dd-9b95-310bc9c6c81b-dshm\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:47:20.371221 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.371192    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"]
Sep 11 19:47:20.376276 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.376235    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-19859a/rate-test-model-19859a-kserve-584dc57655-mgjqz"]
Sep 11 19:47:20.547921 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:20.547889    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f9e1503-9c09-49dd-9b95-310bc9c6c81b" path="/var/lib/kubelet/pods/2f9e1503-9c09-49dd-9b95-310bc9c6c81b/volumes"
Sep 11 19:47:34.346656 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:34.346618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:34.452295 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:34.452235    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:35.103667 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:35.103638    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:35.204070 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:35.204033    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:35.398878 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:35.398789    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:35.523748 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:35.523693    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:36.020260 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:36.020209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:36.741069 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:36.741038    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:37.742964 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:37.742917    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:38.499067 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:38.499012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:38.584743 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:38.584706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:39.333048 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:39.333016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:40.024933 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.024892    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:40.748780 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.748746    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"]
Sep 11 19:47:40.749149 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.749128    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f9e1503-9c09-49dd-9b95-310bc9c6c81b" containerName="main"
Sep 11 19:47:40.749149 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.749138    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f9e1503-9c09-49dd-9b95-310bc9c6c81b" containerName="main"
Sep 11 19:47:40.749227 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.749203    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f9e1503-9c09-49dd-9b95-310bc9c6c81b" containerName="main"
Sep 11 19:47:40.753629 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.753610    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.755890 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.755867    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-904b94-serving-cert\""
Sep 11 19:47:40.762878 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.762855    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"]
Sep 11 19:47:40.780840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.780808    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:40.843104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.843067    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkfdf\" (UniqueName: \"kubernetes.io/projected/7381b624-acc3-4486-b0df-27ee204feb3e-kube-api-access-zkfdf\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.843318 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.843132    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-metrics-certs\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.843318 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.843224    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.843318 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.843294    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7381b624-acc3-4486-b0df-27ee204feb3e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.902119 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.902088    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:40.944200 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.944166    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.944398 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.944207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7381b624-acc3-4486-b0df-27ee204feb3e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.944398 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.944317    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkfdf\" (UniqueName: \"kubernetes.io/projected/7381b624-acc3-4486-b0df-27ee204feb3e-kube-api-access-zkfdf\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.944398 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.944380    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-metrics-certs\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.944857 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.944832    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7381b624-acc3-4486-b0df-27ee204feb3e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.946898 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.946877    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.946999 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.946935    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-metrics-certs\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:40.952712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:40.952692    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkfdf\" (UniqueName: \"kubernetes.io/projected/7381b624-acc3-4486-b0df-27ee204feb3e-kube-api-access-zkfdf\") pod \"maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:41.003651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:41.003572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:41.066428 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:41.066397    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:41.252027 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:41.251983    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:41.333824 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:41.333744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:47:41.343594 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:41.343572    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"]
Sep 11 19:47:41.346293 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:47:41.346265    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7381b624_acc3_4486_b0df_27ee204feb3e.slice/crio-9103d5eb1ccd228024a8a7f75f471e9b6b6a1aa38b433429c6721ea1ba4296b5 WatchSource:0}: Error finding container 9103d5eb1ccd228024a8a7f75f471e9b6b6a1aa38b433429c6721ea1ba4296b5: Status 404 returned error can't find the container with id 9103d5eb1ccd228024a8a7f75f471e9b6b6a1aa38b433429c6721ea1ba4296b5
Sep 11 19:47:41.347787 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:41.347753    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 19:47:42.152285 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:42.152235    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl" event={"ID":"7381b624-acc3-4486-b0df-27ee204feb3e","Type":"ContainerStarted","Data":"45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768"}
Sep 11 19:47:42.152795 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:42.152298    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl" event={"ID":"7381b624-acc3-4486-b0df-27ee204feb3e","Type":"ContainerStarted","Data":"9103d5eb1ccd228024a8a7f75f471e9b6b6a1aa38b433429c6721ea1ba4296b5"}
Sep 11 19:47:42.152795 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:42.152394    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:47:42.170180 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:42.170123    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl" podStartSLOduration=1.91310803 podStartE2EDuration="2.170105714s" totalImagesPullingTime="256.997684ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:47:42.168544464 +0000 UTC m=+1766.097169364" watchObservedRunningTime="2026-09-11 19:47:42.170105714 +0000 UTC m=+1766.098730612"
Sep 11 19:47:48.162670 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:47:48.162639    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:48:02.677816 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:02.677778    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:03.352101 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:03.352060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:03.463927 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:03.463887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:04.156678 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:04.156647    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:04.934215 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:04.934184    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:05.608193 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:05.608156    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:08.201350 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:08.201281    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6st4w_4a7b30ce-04ec-43b8-acff-7ce0680ca302/payload-processing/0.log"
Sep 11 19:48:16.518115 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:16.518079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:30.121553 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:30.121507    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:48.638927 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:48.638887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:48.739455 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:48.739415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:49.291573 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:49.291545    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6st4w_4a7b30ce-04ec-43b8-acff-7ce0680ca302/payload-processing/0.log"
Sep 11 19:48:49.662531 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:49.662496    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:50.010482 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:50.010384    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:50.185416 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:50.185383    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:52.503480 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:52.503450    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:52.735642 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:52.735605    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:52.833983 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:52.833894    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:53.261927 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:53.261880    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:54.248415 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:54.248380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:55.079908 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.079812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:55.772488 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.772457    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"]
Sep 11 19:48:55.776046 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.776021    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.779568 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.779537    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-40573d-serving-cert\""
Sep 11 19:48:55.786184 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.786163    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"]
Sep 11 19:48:55.810334 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.810301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:55.861811 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.861770    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78378756-56dd-4304-8c00-d126ed20aeb1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.861811 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.861813    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c54kf\" (UniqueName: \"kubernetes.io/projected/78378756-56dd-4304-8c00-d126ed20aeb1-kube-api-access-c54kf\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.862043 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.861836    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.862043 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.861911    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.911564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.911527    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:55.962580 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.962539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.962757 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.962649    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78378756-56dd-4304-8c00-d126ed20aeb1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.962757 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.962672    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c54kf\" (UniqueName: \"kubernetes.io/projected/78378756-56dd-4304-8c00-d126ed20aeb1-kube-api-access-c54kf\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.962757 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.962691    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.963429 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.963405    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78378756-56dd-4304-8c00-d126ed20aeb1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.965267 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.965213    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.965384 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.965266    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:55.970359 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:55.970341    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c54kf\" (UniqueName: \"kubernetes.io/projected/78378756-56dd-4304-8c00-d126ed20aeb1-kube-api-access-c54kf\") pod \"maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:56.010958 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.010926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:56.089730 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.089635    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:56.279332 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.279294    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:56.361616 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.361585    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"]
Sep 11 19:48:56.363505 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.363476    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:48:56.364411 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:48:56.364387    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78378756_56dd_4304_8c00_d126ed20aeb1.slice/crio-54f587278dc220bcda322cb802e60ae35918f8bb19e88f4bf523502933bd32ec WatchSource:0}: Error finding container 54f587278dc220bcda322cb802e60ae35918f8bb19e88f4bf523502933bd32ec: Status 404 returned error can't find the container with id 54f587278dc220bcda322cb802e60ae35918f8bb19e88f4bf523502933bd32ec
Sep 11 19:48:56.514681 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.514645    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm" event={"ID":"78378756-56dd-4304-8c00-d126ed20aeb1","Type":"ContainerStarted","Data":"54f587278dc220bcda322cb802e60ae35918f8bb19e88f4bf523502933bd32ec"}
Sep 11 19:48:56.581784 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.581749    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"]
Sep 11 19:48:56.586289 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.586267    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:48:56.588681 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.588658    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-40573d\""
Sep 11 19:48:56.588814 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.588684    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-40573d-dockercfg-vr46f\""
Sep 11 19:48:56.596424 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.596405    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"]
Sep 11 19:48:56.671401 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.671314    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91580031-4554-4007-9f85-cb9457078723-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv\" (UID: \"91580031-4554-4007-9f85-cb9457078723\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:48:56.671401 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.671359    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c49w2\" (UniqueName: \"kubernetes.io/projected/91580031-4554-4007-9f85-cb9457078723-kube-api-access-c49w2\") pod \"payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv\" (UID: \"91580031-4554-4007-9f85-cb9457078723\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:48:56.772754 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.772722    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91580031-4554-4007-9f85-cb9457078723-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv\" (UID: \"91580031-4554-4007-9f85-cb9457078723\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:48:56.773055 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.772779    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c49w2\" (UniqueName: \"kubernetes.io/projected/91580031-4554-4007-9f85-cb9457078723-kube-api-access-c49w2\") pod \"payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv\" (UID: \"91580031-4554-4007-9f85-cb9457078723\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:48:56.773473 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.773456    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91580031-4554-4007-9f85-cb9457078723-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv\" (UID: \"91580031-4554-4007-9f85-cb9457078723\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:48:56.781162 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.781136    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c49w2\" (UniqueName: \"kubernetes.io/projected/91580031-4554-4007-9f85-cb9457078723-kube-api-access-c49w2\") pod \"payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv\" (UID: \"91580031-4554-4007-9f85-cb9457078723\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:48:56.899634 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:56.899607    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:48:57.174997 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:57.174807    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"]
Sep 11 19:48:57.177599 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:48:57.177563    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91580031_4554_4007_9f85_cb9457078723.slice/crio-cfda3b3bb3402983f8b13a7fc7a0365d7118116a5af89ae7f4c5830f12c551f7 WatchSource:0}: Error finding container cfda3b3bb3402983f8b13a7fc7a0365d7118116a5af89ae7f4c5830f12c551f7: Status 404 returned error can't find the container with id cfda3b3bb3402983f8b13a7fc7a0365d7118116a5af89ae7f4c5830f12c551f7
Sep 11 19:48:57.521506 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:57.521402    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm" event={"ID":"78378756-56dd-4304-8c00-d126ed20aeb1","Type":"ContainerStarted","Data":"52cb7d070274f8df697e8ed0f604f24fe81561c1a7cfa3b2f59bb8b418fa88db"}
Sep 11 19:48:57.521506 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:57.521469    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:48:57.523071 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:57.523050    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv" event={"ID":"91580031-4554-4007-9f85-cb9457078723","Type":"ContainerStarted","Data":"4c726f81a1e1b1c6a551b3b44a841f72bacf570cb5e84a47ee44bab2d1add91e"}
Sep 11 19:48:57.523201 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:57.523085    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv" event={"ID":"91580031-4554-4007-9f85-cb9457078723","Type":"ContainerStarted","Data":"cfda3b3bb3402983f8b13a7fc7a0365d7118116a5af89ae7f4c5830f12c551f7"}
Sep 11 19:48:57.523201 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:57.523193    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:48:57.537418 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:57.537354    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm" podStartSLOduration=2.196966056 podStartE2EDuration="2.537336831s" totalImagesPullingTime="340.370775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:48:57.536824302 +0000 UTC m=+1841.465449201" watchObservedRunningTime="2026-09-11 19:48:57.537336831 +0000 UTC m=+1841.465961730"
Sep 11 19:48:57.554514 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:48:57.554464    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv" podStartSLOduration=1.554448427 podStartE2EDuration="1.554448427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:48:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:48:57.552747198 +0000 UTC m=+1841.481372097" watchObservedRunningTime="2026-09-11 19:48:57.554448427 +0000 UTC m=+1841.483073325"
Sep 11 19:49:03.541407 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:03.541377    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:49:08.538290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:08.538254    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:49:18.828055 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:18.828019    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:19.275207 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:19.275170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:19.484255 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:19.484217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:20.494236 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:20.494200    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:20.920048 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:20.920011    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:21.567104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:21.567066    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:22.342129 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:22.342093    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:22.892171 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:22.892133    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:24.327822 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:24.327788    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:25.151348 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:25.151299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:25.825621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:25.825561    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:26.637149 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:26.637114    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:27.351985 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:27.351934    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:28.032114 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:28.032087    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:34.411730 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.411695    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"]
Sep 11 19:49:34.412215 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.411916    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm" podUID="78378756-56dd-4304-8c00-d126ed20aeb1" containerName="maas-api" containerID="cri-o://52cb7d070274f8df697e8ed0f604f24fe81561c1a7cfa3b2f59bb8b418fa88db" gracePeriod=30
Sep 11 19:49:34.439562 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.439520    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:34.467585 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.466933    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"]
Sep 11 19:49:34.467585 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.467175    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv" podUID="91580031-4554-4007-9f85-cb9457078723" containerName="payload-processing" containerID="cri-o://4c726f81a1e1b1c6a551b3b44a841f72bacf570cb5e84a47ee44bab2d1add91e" gracePeriod=30
Sep 11 19:49:34.509806 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.509773    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:34.610727 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.610698    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:34.677737 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.677637    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:34.718013 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.717984    2583 generic.go:356] "Generic (PLEG): container finished" podID="78378756-56dd-4304-8c00-d126ed20aeb1" containerID="52cb7d070274f8df697e8ed0f604f24fe81561c1a7cfa3b2f59bb8b418fa88db" exitCode=0
Sep 11 19:49:34.718148 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.718054    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm" event={"ID":"78378756-56dd-4304-8c00-d126ed20aeb1","Type":"ContainerDied","Data":"52cb7d070274f8df697e8ed0f604f24fe81561c1a7cfa3b2f59bb8b418fa88db"}
Sep 11 19:49:34.718148 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.718122    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm" event={"ID":"78378756-56dd-4304-8c00-d126ed20aeb1","Type":"ContainerDied","Data":"54f587278dc220bcda322cb802e60ae35918f8bb19e88f4bf523502933bd32ec"}
Sep 11 19:49:34.718148 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.718145    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54f587278dc220bcda322cb802e60ae35918f8bb19e88f4bf523502933bd32ec"
Sep 11 19:49:34.720082 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.720054    2583 generic.go:356] "Generic (PLEG): container finished" podID="91580031-4554-4007-9f85-cb9457078723" containerID="4c726f81a1e1b1c6a551b3b44a841f72bacf570cb5e84a47ee44bab2d1add91e" exitCode=0
Sep 11 19:49:34.720212 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.720138    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv" event={"ID":"91580031-4554-4007-9f85-cb9457078723","Type":"ContainerDied","Data":"4c726f81a1e1b1c6a551b3b44a841f72bacf570cb5e84a47ee44bab2d1add91e"}
Sep 11 19:49:34.732636 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.732612    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:49:34.765199 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.765169    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:49:34.842989 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.842944    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-metrics-certs\") pod \"78378756-56dd-4304-8c00-d126ed20aeb1\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") "
Sep 11 19:49:34.842989 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.842984    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-maas-api-tls\") pod \"78378756-56dd-4304-8c00-d126ed20aeb1\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") "
Sep 11 19:49:34.843202 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.843019    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c54kf\" (UniqueName: \"kubernetes.io/projected/78378756-56dd-4304-8c00-d126ed20aeb1-kube-api-access-c54kf\") pod \"78378756-56dd-4304-8c00-d126ed20aeb1\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") "
Sep 11 19:49:34.843202 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.843088    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c49w2\" (UniqueName: \"kubernetes.io/projected/91580031-4554-4007-9f85-cb9457078723-kube-api-access-c49w2\") pod \"91580031-4554-4007-9f85-cb9457078723\" (UID: \"91580031-4554-4007-9f85-cb9457078723\") "
Sep 11 19:49:34.843202 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.843155    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78378756-56dd-4304-8c00-d126ed20aeb1-openshift-service-ca\") pod \"78378756-56dd-4304-8c00-d126ed20aeb1\" (UID: \"78378756-56dd-4304-8c00-d126ed20aeb1\") "
Sep 11 19:49:34.843202 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.843192    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91580031-4554-4007-9f85-cb9457078723-plugins-config-volume\") pod \"91580031-4554-4007-9f85-cb9457078723\" (UID: \"91580031-4554-4007-9f85-cb9457078723\") "
Sep 11 19:49:34.843621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.843591    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78378756-56dd-4304-8c00-d126ed20aeb1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "78378756-56dd-4304-8c00-d126ed20aeb1" (UID: "78378756-56dd-4304-8c00-d126ed20aeb1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:49:34.843938 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.843608    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91580031-4554-4007-9f85-cb9457078723-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "91580031-4554-4007-9f85-cb9457078723" (UID: "91580031-4554-4007-9f85-cb9457078723"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:49:34.845508 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.845480    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78378756-56dd-4304-8c00-d126ed20aeb1-kube-api-access-c54kf" (OuterVolumeSpecName: "kube-api-access-c54kf") pod "78378756-56dd-4304-8c00-d126ed20aeb1" (UID: "78378756-56dd-4304-8c00-d126ed20aeb1"). InnerVolumeSpecName "kube-api-access-c54kf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:49:34.845508 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.845483    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "78378756-56dd-4304-8c00-d126ed20aeb1" (UID: "78378756-56dd-4304-8c00-d126ed20aeb1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:49:34.845660 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.845530    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "78378756-56dd-4304-8c00-d126ed20aeb1" (UID: "78378756-56dd-4304-8c00-d126ed20aeb1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:49:34.845660 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.845567    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91580031-4554-4007-9f85-cb9457078723-kube-api-access-c49w2" (OuterVolumeSpecName: "kube-api-access-c49w2") pod "91580031-4554-4007-9f85-cb9457078723" (UID: "91580031-4554-4007-9f85-cb9457078723"). InnerVolumeSpecName "kube-api-access-c49w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:49:34.944081 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.944000    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:49:34.944081 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.944030    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78378756-56dd-4304-8c00-d126ed20aeb1-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:49:34.944081 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.944039    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c54kf\" (UniqueName: \"kubernetes.io/projected/78378756-56dd-4304-8c00-d126ed20aeb1-kube-api-access-c54kf\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:49:34.944081 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.944050    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c49w2\" (UniqueName: \"kubernetes.io/projected/91580031-4554-4007-9f85-cb9457078723-kube-api-access-c49w2\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:49:34.944081 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.944058    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78378756-56dd-4304-8c00-d126ed20aeb1-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:49:34.944081 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:34.944066    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91580031-4554-4007-9f85-cb9457078723-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:49:35.037291 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:35.037259    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:35.139333 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:35.139289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:35.726477 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:35.726382    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv" event={"ID":"91580031-4554-4007-9f85-cb9457078723","Type":"ContainerDied","Data":"cfda3b3bb3402983f8b13a7fc7a0365d7118116a5af89ae7f4c5830f12c551f7"}
Sep 11 19:49:35.726477 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:35.726412    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"
Sep 11 19:49:35.726477 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:35.726423    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"
Sep 11 19:49:35.726477 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:35.726461    2583 scope.go:117] "RemoveContainer" containerID="4c726f81a1e1b1c6a551b3b44a841f72bacf570cb5e84a47ee44bab2d1add91e"
Sep 11 19:49:35.751256 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:35.751209    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"]
Sep 11 19:49:35.756318 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:35.756295    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-40573d-9949fb49c-nztsm"]
Sep 11 19:49:35.760798 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:35.760775    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:35.766519 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:35.766498    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"]
Sep 11 19:49:35.770303 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:35.770280    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-40573d-5fd6674b65-8hqjv"]
Sep 11 19:49:36.546195 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:36.546151    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78378756-56dd-4304-8c00-d126ed20aeb1" path="/var/lib/kubelet/pods/78378756-56dd-4304-8c00-d126ed20aeb1/volumes"
Sep 11 19:49:36.546625 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:36.546604    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91580031-4554-4007-9f85-cb9457078723" path="/var/lib/kubelet/pods/91580031-4554-4007-9f85-cb9457078723/volumes"
Sep 11 19:49:36.633345 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:36.633310    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:37.371666 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:37.371636    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:38.033309 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:38.033265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:41.102990 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:41.102957    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:41.170075 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:41.170040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:41.364130 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:41.364045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:42.122815 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:42.122777    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:42.259421 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:42.259349    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:45.002330 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:45.002292    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:45.759017 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:45.758985    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:46.552032 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:46.552002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:47.253789 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:47.253755    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:48.034562 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:48.034525    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:49:48.753150 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:49:48.753111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:00.292170 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.292132    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"]
Sep 11 19:50:00.292801 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.292371    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf" podUID="fd0288d1-15eb-4c2e-acff-b821666d171c" containerName="maas-api" containerID="cri-o://d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64" gracePeriod=30
Sep 11 19:50:00.319683 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.319654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:00.393571 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.393538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:00.455387 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.455354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:00.533568 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.533534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:00.543115 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.543045    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:50:00.593078 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.593044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:00.694687 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.694657    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-maas-api-tls\") pod \"fd0288d1-15eb-4c2e-acff-b821666d171c\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") "
Sep 11 19:50:00.694847 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.694714    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szs6q\" (UniqueName: \"kubernetes.io/projected/fd0288d1-15eb-4c2e-acff-b821666d171c-kube-api-access-szs6q\") pod \"fd0288d1-15eb-4c2e-acff-b821666d171c\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") "
Sep 11 19:50:00.694847 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.694770    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-metrics-certs\") pod \"fd0288d1-15eb-4c2e-acff-b821666d171c\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") "
Sep 11 19:50:00.694847 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.694839    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd0288d1-15eb-4c2e-acff-b821666d171c-openshift-service-ca\") pod \"fd0288d1-15eb-4c2e-acff-b821666d171c\" (UID: \"fd0288d1-15eb-4c2e-acff-b821666d171c\") "
Sep 11 19:50:00.695262 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.695212    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd0288d1-15eb-4c2e-acff-b821666d171c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd0288d1-15eb-4c2e-acff-b821666d171c" (UID: "fd0288d1-15eb-4c2e-acff-b821666d171c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:50:00.696966 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.696937    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd0288d1-15eb-4c2e-acff-b821666d171c" (UID: "fd0288d1-15eb-4c2e-acff-b821666d171c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:50:00.697066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.696968    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd0288d1-15eb-4c2e-acff-b821666d171c-kube-api-access-szs6q" (OuterVolumeSpecName: "kube-api-access-szs6q") pod "fd0288d1-15eb-4c2e-acff-b821666d171c" (UID: "fd0288d1-15eb-4c2e-acff-b821666d171c"). InnerVolumeSpecName "kube-api-access-szs6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:50:00.697066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.696975    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fd0288d1-15eb-4c2e-acff-b821666d171c" (UID: "fd0288d1-15eb-4c2e-acff-b821666d171c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:50:00.796311 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.796204    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd0288d1-15eb-4c2e-acff-b821666d171c-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:50:00.796311 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.796234    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:50:00.796311 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.796266    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szs6q\" (UniqueName: \"kubernetes.io/projected/fd0288d1-15eb-4c2e-acff-b821666d171c-kube-api-access-szs6q\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:50:00.796311 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.796275    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd0288d1-15eb-4c2e-acff-b821666d171c-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:50:00.851571 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.851537    2583 generic.go:356] "Generic (PLEG): container finished" podID="fd0288d1-15eb-4c2e-acff-b821666d171c" containerID="d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64" exitCode=0
Sep 11 19:50:00.851721 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.851586    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"
Sep 11 19:50:00.851721 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.851628    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf" event={"ID":"fd0288d1-15eb-4c2e-acff-b821666d171c","Type":"ContainerDied","Data":"d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64"}
Sep 11 19:50:00.851721 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.851680    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf" event={"ID":"fd0288d1-15eb-4c2e-acff-b821666d171c","Type":"ContainerDied","Data":"c5f851004ba4d15158b0de357b7828072bc41a705d7e0f7e6273b31e05105ad4"}
Sep 11 19:50:00.851721 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.851704    2583 scope.go:117] "RemoveContainer" containerID="d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64"
Sep 11 19:50:00.861914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.861896    2583 scope.go:117] "RemoveContainer" containerID="d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64"
Sep 11 19:50:00.862212 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:50:00.862192    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64\": container with ID starting with d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64 not found: ID does not exist" containerID="d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64"
Sep 11 19:50:00.862310 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.862226    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64"} err="failed to get container status \"d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64\": rpc error: code = NotFound desc = could not find container \"d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64\": container with ID starting with d274993bb41f09b0e218484bb7eb05b8cfc8479d17ce1ad1afcf8c23f3b38e64 not found: ID does not exist"
Sep 11 19:50:00.873826 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.873793    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"]
Sep 11 19:50:00.878649 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.878621    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0a42d-86685bd467-r75vf"]
Sep 11 19:50:00.899636 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:00.899607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:01.002862 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:01.002826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:01.615089 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:01.615049    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:02.300947 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:02.300912    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:02.546696 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:02.546662    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd0288d1-15eb-4c2e-acff-b821666d171c" path="/var/lib/kubelet/pods/fd0288d1-15eb-4c2e-acff-b821666d171c/volumes"
Sep 11 19:50:03.150072 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:03.150035    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:03.777925 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:03.777894    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:05.969885 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:05.969851    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:29.301022 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:29.300988    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:29.859730 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:29.859694    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:38.149549 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:38.149516    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:40.395209 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:40.395171    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:41.057296 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:41.057261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:42.766642 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:42.766600    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:42.943996 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:42.943966    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:48.422300 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:48.422261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:54.103121 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:54.103078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:54.166261 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:54.166190    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:54.368022 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:54.367939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:54.848125 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:54.848087    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:55.634440 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:55.634402    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:56.547959 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:56.547924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:57.245806 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:57.245772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:50:58.197725 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:50:58.197690    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:05.243899 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.243866    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"]
Sep 11 19:51:05.244385 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.244094    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl" podUID="7381b624-acc3-4486-b0df-27ee204feb3e" containerName="maas-api" containerID="cri-o://45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768" gracePeriod=30
Sep 11 19:51:05.272868 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.272830    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:05.338654 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.338624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:05.396538 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.396504    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:05.469390 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.469362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:05.488297 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.488276    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:51:05.541051 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.540975    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:05.585899 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.585868    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-maas-api-tls\") pod \"7381b624-acc3-4486-b0df-27ee204feb3e\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") "
Sep 11 19:51:05.586073 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.585926    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-metrics-certs\") pod \"7381b624-acc3-4486-b0df-27ee204feb3e\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") "
Sep 11 19:51:05.586073 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.586041    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7381b624-acc3-4486-b0df-27ee204feb3e-openshift-service-ca\") pod \"7381b624-acc3-4486-b0df-27ee204feb3e\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") "
Sep 11 19:51:05.586191 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.586076    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkfdf\" (UniqueName: \"kubernetes.io/projected/7381b624-acc3-4486-b0df-27ee204feb3e-kube-api-access-zkfdf\") pod \"7381b624-acc3-4486-b0df-27ee204feb3e\" (UID: \"7381b624-acc3-4486-b0df-27ee204feb3e\") "
Sep 11 19:51:05.586483 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.586442    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7381b624-acc3-4486-b0df-27ee204feb3e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7381b624-acc3-4486-b0df-27ee204feb3e" (UID: "7381b624-acc3-4486-b0df-27ee204feb3e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:51:05.588746 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.588714    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7381b624-acc3-4486-b0df-27ee204feb3e" (UID: "7381b624-acc3-4486-b0df-27ee204feb3e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:51:05.588871 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.588748    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7381b624-acc3-4486-b0df-27ee204feb3e" (UID: "7381b624-acc3-4486-b0df-27ee204feb3e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:51:05.589349 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.589325    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7381b624-acc3-4486-b0df-27ee204feb3e-kube-api-access-zkfdf" (OuterVolumeSpecName: "kube-api-access-zkfdf") pod "7381b624-acc3-4486-b0df-27ee204feb3e" (UID: "7381b624-acc3-4486-b0df-27ee204feb3e"). InnerVolumeSpecName "kube-api-access-zkfdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:51:05.687802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.687760    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:05.687802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.687798    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7381b624-acc3-4486-b0df-27ee204feb3e-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:05.688023 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.687813    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkfdf\" (UniqueName: \"kubernetes.io/projected/7381b624-acc3-4486-b0df-27ee204feb3e-kube-api-access-zkfdf\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:05.688023 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.687826    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7381b624-acc3-4486-b0df-27ee204feb3e-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:05.825831 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.825730    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:05.926380 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:05.926341    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:06.163269 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:06.163210    2583 generic.go:356] "Generic (PLEG): container finished" podID="7381b624-acc3-4486-b0df-27ee204feb3e" containerID="45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768" exitCode=0
Sep 11 19:51:06.163269 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:06.163271    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"
Sep 11 19:51:06.163481 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:06.163283    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl" event={"ID":"7381b624-acc3-4486-b0df-27ee204feb3e","Type":"ContainerDied","Data":"45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768"}
Sep 11 19:51:06.163481 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:06.163343    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl" event={"ID":"7381b624-acc3-4486-b0df-27ee204feb3e","Type":"ContainerDied","Data":"9103d5eb1ccd228024a8a7f75f471e9b6b6a1aa38b433429c6721ea1ba4296b5"}
Sep 11 19:51:06.163481 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:06.163377    2583 scope.go:117] "RemoveContainer" containerID="45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768"
Sep 11 19:51:06.172809 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:06.172789    2583 scope.go:117] "RemoveContainer" containerID="45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768"
Sep 11 19:51:06.173097 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:51:06.173078    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768\": container with ID starting with 45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768 not found: ID does not exist" containerID="45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768"
Sep 11 19:51:06.173178 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:06.173106    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768"} err="failed to get container status \"45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768\": rpc error: code = NotFound desc = could not find container \"45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768\": container with ID starting with 45044f5bdc2c7f64e462dedeaf6c7941ec71bc4a740ce38f1311b0721c156768 not found: ID does not exist"
Sep 11 19:51:06.190934 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:06.190905    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"]
Sep 11 19:51:06.196003 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:06.195972    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-904b94-6d75cd57cb-zgnhl"]
Sep 11 19:51:06.546153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:06.546044    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7381b624-acc3-4486-b0df-27ee204feb3e" path="/var/lib/kubelet/pods/7381b624-acc3-4486-b0df-27ee204feb3e/volumes"
Sep 11 19:51:06.733641 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:06.733598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:07.547358 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:07.547330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:11.588760 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:11.588724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:11.698400 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:11.698365    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:11.872433 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:11.872399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:12.925759 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:12.925726    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:13.747414 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:13.747379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:14.429664 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:14.429622    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:22.700007 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:22.699970    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:22.735315 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:22.735279    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"]
Sep 11 19:51:22.735549 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:22.735494    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs" podUID="1af2b69e-5830-40bf-87e6-81efaded35ad" containerName="payload-pre-processing" containerID="cri-o://f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5" gracePeriod=30
Sep 11 19:51:22.800997 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:22.800965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:22.981153 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:22.981121    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:23.007630 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.007605    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:51:23.040810 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.040783    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:23.155032 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.155000    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1af2b69e-5830-40bf-87e6-81efaded35ad-plugins-config-volume\") pod \"1af2b69e-5830-40bf-87e6-81efaded35ad\" (UID: \"1af2b69e-5830-40bf-87e6-81efaded35ad\") "
Sep 11 19:51:23.155290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.155041    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-df55p\" (UniqueName: \"kubernetes.io/projected/1af2b69e-5830-40bf-87e6-81efaded35ad-kube-api-access-df55p\") pod \"1af2b69e-5830-40bf-87e6-81efaded35ad\" (UID: \"1af2b69e-5830-40bf-87e6-81efaded35ad\") "
Sep 11 19:51:23.155504 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.155480    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1af2b69e-5830-40bf-87e6-81efaded35ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1af2b69e-5830-40bf-87e6-81efaded35ad" (UID: "1af2b69e-5830-40bf-87e6-81efaded35ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:51:23.157447 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.157425    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1af2b69e-5830-40bf-87e6-81efaded35ad-kube-api-access-df55p" (OuterVolumeSpecName: "kube-api-access-df55p") pod "1af2b69e-5830-40bf-87e6-81efaded35ad" (UID: "1af2b69e-5830-40bf-87e6-81efaded35ad"). InnerVolumeSpecName "kube-api-access-df55p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:51:23.248770 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.248671    2583 generic.go:356] "Generic (PLEG): container finished" podID="1af2b69e-5830-40bf-87e6-81efaded35ad" containerID="f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5" exitCode=0
Sep 11 19:51:23.248770 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.248735    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"
Sep 11 19:51:23.248770 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.248758    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs" event={"ID":"1af2b69e-5830-40bf-87e6-81efaded35ad","Type":"ContainerDied","Data":"f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5"}
Sep 11 19:51:23.249021 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.248807    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs" event={"ID":"1af2b69e-5830-40bf-87e6-81efaded35ad","Type":"ContainerDied","Data":"5f8928c7a156cd79b5dc1c711b512372e866963142b20c2a782e5918a9d47f1d"}
Sep 11 19:51:23.249021 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.248831    2583 scope.go:117] "RemoveContainer" containerID="f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5"
Sep 11 19:51:23.255802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.255765    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1af2b69e-5830-40bf-87e6-81efaded35ad-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:23.255802 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.255801    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-df55p\" (UniqueName: \"kubernetes.io/projected/1af2b69e-5830-40bf-87e6-81efaded35ad-kube-api-access-df55p\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:23.257824 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.257776    2583 scope.go:117] "RemoveContainer" containerID="f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5"
Sep 11 19:51:23.258083 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:51:23.258065    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5\": container with ID starting with f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5 not found: ID does not exist" containerID="f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5"
Sep 11 19:51:23.258179 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.258092    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5"} err="failed to get container status \"f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5\": rpc error: code = NotFound desc = could not find container \"f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5\": container with ID starting with f1681cf465d61e788d568f3130be8329a0835e1429fcd9e748294dfef69dc2c5 not found: ID does not exist"
Sep 11 19:51:23.272410 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.272383    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"]
Sep 11 19:51:23.276050 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.276027    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-19859a-5bd5f78cd-64zzs"]
Sep 11 19:51:23.363441 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.363407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:23.465795 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:23.465755    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:24.112321 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:24.112283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:24.547091 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:24.547013    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1af2b69e-5830-40bf-87e6-81efaded35ad" path="/var/lib/kubelet/pods/1af2b69e-5830-40bf-87e6-81efaded35ad/volumes"
Sep 11 19:51:29.167905 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:29.167864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:29.223967 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:29.223937    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:29.439087 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:29.439003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:30.564722 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:30.564691    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:31.550742 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:31.550699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:40.238224 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.238191    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"]
Sep 11 19:51:40.238750 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.238417    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s" podUID="283f6644-486d-4878-9545-8536d99f4ad5" containerName="maas-api" containerID="cri-o://fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587" gracePeriod=30
Sep 11 19:51:40.266466 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.266437    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:40.288458 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.288431    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"]
Sep 11 19:51:40.288710 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.288659    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz" podUID="06869ae7-9c43-4b03-9b32-490b86d7dba9" containerName="payload-processing" containerID="cri-o://8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a" gracePeriod=30
Sep 11 19:51:40.295850 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.295817    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"]
Sep 11 19:51:40.296043 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.296017    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws" podUID="8e1024cb-256f-4317-8f29-a0ef4f0b491a" containerName="payload-pre-processing" containerID="cri-o://f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0" gracePeriod=30
Sep 11 19:51:40.328875 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.328849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:40.385501 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.385464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:40.464608 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.461616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:40.532870 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.532782    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:40.556683 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.556661    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:51:40.559399 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.559370    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:51:40.597546 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.597520    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:51:40.619197 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.619155    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-655ts\" (UniqueName: \"kubernetes.io/projected/06869ae7-9c43-4b03-9b32-490b86d7dba9-kube-api-access-655ts\") pod \"06869ae7-9c43-4b03-9b32-490b86d7dba9\" (UID: \"06869ae7-9c43-4b03-9b32-490b86d7dba9\") "
Sep 11 19:51:40.619513 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.619492    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6668\" (UniqueName: \"kubernetes.io/projected/8e1024cb-256f-4317-8f29-a0ef4f0b491a-kube-api-access-v6668\") pod \"8e1024cb-256f-4317-8f29-a0ef4f0b491a\" (UID: \"8e1024cb-256f-4317-8f29-a0ef4f0b491a\") "
Sep 11 19:51:40.619634 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.619619    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-maas-api-tls\") pod \"283f6644-486d-4878-9545-8536d99f4ad5\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") "
Sep 11 19:51:40.619746 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.619734    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1024cb-256f-4317-8f29-a0ef4f0b491a-plugins-config-volume\") pod \"8e1024cb-256f-4317-8f29-a0ef4f0b491a\" (UID: \"8e1024cb-256f-4317-8f29-a0ef4f0b491a\") "
Sep 11 19:51:40.619862 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.619849    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-metrics-certs\") pod \"283f6644-486d-4878-9545-8536d99f4ad5\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") "
Sep 11 19:51:40.619973 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.619959    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/283f6644-486d-4878-9545-8536d99f4ad5-openshift-service-ca\") pod \"283f6644-486d-4878-9545-8536d99f4ad5\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") "
Sep 11 19:51:40.620090 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.620079    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06869ae7-9c43-4b03-9b32-490b86d7dba9-plugins-config-volume\") pod \"06869ae7-9c43-4b03-9b32-490b86d7dba9\" (UID: \"06869ae7-9c43-4b03-9b32-490b86d7dba9\") "
Sep 11 19:51:40.620190 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.620178    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnl7r\" (UniqueName: \"kubernetes.io/projected/283f6644-486d-4878-9545-8536d99f4ad5-kube-api-access-bnl7r\") pod \"283f6644-486d-4878-9545-8536d99f4ad5\" (UID: \"283f6644-486d-4878-9545-8536d99f4ad5\") "
Sep 11 19:51:40.623040 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.622746    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e1024cb-256f-4317-8f29-a0ef4f0b491a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e1024cb-256f-4317-8f29-a0ef4f0b491a" (UID: "8e1024cb-256f-4317-8f29-a0ef4f0b491a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:51:40.627807 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.627777    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06869ae7-9c43-4b03-9b32-490b86d7dba9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "06869ae7-9c43-4b03-9b32-490b86d7dba9" (UID: "06869ae7-9c43-4b03-9b32-490b86d7dba9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:51:40.628030 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.627777    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/283f6644-486d-4878-9545-8536d99f4ad5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "283f6644-486d-4878-9545-8536d99f4ad5" (UID: "283f6644-486d-4878-9545-8536d99f4ad5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:51:40.628360 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.627795    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/283f6644-486d-4878-9545-8536d99f4ad5-kube-api-access-bnl7r" (OuterVolumeSpecName: "kube-api-access-bnl7r") pod "283f6644-486d-4878-9545-8536d99f4ad5" (UID: "283f6644-486d-4878-9545-8536d99f4ad5"). InnerVolumeSpecName "kube-api-access-bnl7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:51:40.628923 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.628899    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06869ae7-9c43-4b03-9b32-490b86d7dba9-kube-api-access-655ts" (OuterVolumeSpecName: "kube-api-access-655ts") pod "06869ae7-9c43-4b03-9b32-490b86d7dba9" (UID: "06869ae7-9c43-4b03-9b32-490b86d7dba9"). InnerVolumeSpecName "kube-api-access-655ts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:51:40.629827 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.629801    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e1024cb-256f-4317-8f29-a0ef4f0b491a-kube-api-access-v6668" (OuterVolumeSpecName: "kube-api-access-v6668") pod "8e1024cb-256f-4317-8f29-a0ef4f0b491a" (UID: "8e1024cb-256f-4317-8f29-a0ef4f0b491a"). InnerVolumeSpecName "kube-api-access-v6668". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:51:40.630032 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.630016    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "283f6644-486d-4878-9545-8536d99f4ad5" (UID: "283f6644-486d-4878-9545-8536d99f4ad5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:51:40.630092 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.630060    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "283f6644-486d-4878-9545-8536d99f4ad5" (UID: "283f6644-486d-4878-9545-8536d99f4ad5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:51:40.721916 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.721874    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1024cb-256f-4317-8f29-a0ef4f0b491a-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:40.721916 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.721911    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:40.721916 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.721920    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/283f6644-486d-4878-9545-8536d99f4ad5-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:40.722159 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.721929    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06869ae7-9c43-4b03-9b32-490b86d7dba9-plugins-config-volume\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:40.722159 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.721938    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnl7r\" (UniqueName: \"kubernetes.io/projected/283f6644-486d-4878-9545-8536d99f4ad5-kube-api-access-bnl7r\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:40.722159 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.721946    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-655ts\" (UniqueName: \"kubernetes.io/projected/06869ae7-9c43-4b03-9b32-490b86d7dba9-kube-api-access-655ts\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:40.722159 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.721954    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6668\" (UniqueName: \"kubernetes.io/projected/8e1024cb-256f-4317-8f29-a0ef4f0b491a-kube-api-access-v6668\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:40.722159 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.721963    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/283f6644-486d-4878-9545-8536d99f4ad5-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:40.827108 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.827031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:40.948060 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:40.948006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:41.343175 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.343141    2583 generic.go:356] "Generic (PLEG): container finished" podID="283f6644-486d-4878-9545-8536d99f4ad5" containerID="fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587" exitCode=0
Sep 11 19:51:41.343675 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.343199    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"
Sep 11 19:51:41.343675 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.343222    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s" event={"ID":"283f6644-486d-4878-9545-8536d99f4ad5","Type":"ContainerDied","Data":"fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587"}
Sep 11 19:51:41.343675 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.343325    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s" event={"ID":"283f6644-486d-4878-9545-8536d99f4ad5","Type":"ContainerDied","Data":"1aada969e8bfb8d9455d2d82f32e4cb157581d79fcdbeaeeafed525fa32eedda"}
Sep 11 19:51:41.343675 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.343360    2583 scope.go:117] "RemoveContainer" containerID="fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587"
Sep 11 19:51:41.344835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.344726    2583 generic.go:356] "Generic (PLEG): container finished" podID="06869ae7-9c43-4b03-9b32-490b86d7dba9" containerID="8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a" exitCode=0
Sep 11 19:51:41.344835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.344756    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz" event={"ID":"06869ae7-9c43-4b03-9b32-490b86d7dba9","Type":"ContainerDied","Data":"8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a"}
Sep 11 19:51:41.344835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.344784    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"
Sep 11 19:51:41.344835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.344807    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz" event={"ID":"06869ae7-9c43-4b03-9b32-490b86d7dba9","Type":"ContainerDied","Data":"97062426fab0561b440ce834c84889f80269c5f99a5d1569cc56360945133019"}
Sep 11 19:51:41.346290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.346264    2583 generic.go:356] "Generic (PLEG): container finished" podID="8e1024cb-256f-4317-8f29-a0ef4f0b491a" containerID="f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0" exitCode=0
Sep 11 19:51:41.346417 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.346292    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws" event={"ID":"8e1024cb-256f-4317-8f29-a0ef4f0b491a","Type":"ContainerDied","Data":"f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0"}
Sep 11 19:51:41.346417 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.346323    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"
Sep 11 19:51:41.346417 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.346354    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws" event={"ID":"8e1024cb-256f-4317-8f29-a0ef4f0b491a","Type":"ContainerDied","Data":"0ba6cf19e948984c21e7a59e77b6bb08eedf17227d81428da3f51ad2a7dde4e9"}
Sep 11 19:51:41.354719 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.354700    2583 scope.go:117] "RemoveContainer" containerID="fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587"
Sep 11 19:51:41.355031 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:51:41.355006    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587\": container with ID starting with fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587 not found: ID does not exist" containerID="fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587"
Sep 11 19:51:41.355298 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.355035    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587"} err="failed to get container status \"fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587\": rpc error: code = NotFound desc = could not find container \"fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587\": container with ID starting with fadbedca51de108c3d61a9b122966c8a5b8d7eeeaac2dcda16764145a69b0587 not found: ID does not exist"
Sep 11 19:51:41.355298 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.355052    2583 scope.go:117] "RemoveContainer" containerID="8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a"
Sep 11 19:51:41.363555 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.363535    2583 scope.go:117] "RemoveContainer" containerID="8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a"
Sep 11 19:51:41.363872 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:51:41.363850    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a\": container with ID starting with 8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a not found: ID does not exist" containerID="8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a"
Sep 11 19:51:41.363963 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.363878    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a"} err="failed to get container status \"8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a\": rpc error: code = NotFound desc = could not find container \"8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a\": container with ID starting with 8cea107f2225ac5a394fd7ac4f5f13516836dcb8e8945bc3d18d78d67feaa97a not found: ID does not exist"
Sep 11 19:51:41.363963 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.363899    2583 scope.go:117] "RemoveContainer" containerID="f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0"
Sep 11 19:51:41.370186 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.370164    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"]
Sep 11 19:51:41.372786 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.372770    2583 scope.go:117] "RemoveContainer" containerID="f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0"
Sep 11 19:51:41.373086 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:51:41.373068    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0\": container with ID starting with f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0 not found: ID does not exist" containerID="f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0"
Sep 11 19:51:41.373133 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.373093    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0"} err="failed to get container status \"f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0\": rpc error: code = NotFound desc = could not find container \"f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0\": container with ID starting with f4ba9d63c34a60897f88c9c3b4a026ccaeb4945a5519b2fb5c8378acb16bbba0 not found: ID does not exist"
Sep 11 19:51:41.373833 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.373813    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-abcdff-56f79c8dcc-dsh9s"]
Sep 11 19:51:41.387499 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.387472    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"]
Sep 11 19:51:41.389551 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.389527    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-abcdff-765f9b54df-hzdlz"]
Sep 11 19:51:41.399548 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.399524    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"]
Sep 11 19:51:41.405556 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:41.405527    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abcdff-6d668b7ff8-zz5ws"]
Sep 11 19:51:42.546897 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:42.546861    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06869ae7-9c43-4b03-9b32-490b86d7dba9" path="/var/lib/kubelet/pods/06869ae7-9c43-4b03-9b32-490b86d7dba9/volumes"
Sep 11 19:51:42.547461 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:42.547441    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="283f6644-486d-4878-9545-8536d99f4ad5" path="/var/lib/kubelet/pods/283f6644-486d-4878-9545-8536d99f4ad5/volumes"
Sep 11 19:51:42.547820 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:42.547801    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e1024cb-256f-4317-8f29-a0ef4f0b491a" path="/var/lib/kubelet/pods/8e1024cb-256f-4317-8f29-a0ef4f0b491a/volumes"
Sep 11 19:51:46.718700 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:46.718669    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:46.820007 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:46.819977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:46.971816 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:46.971726    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:51:49.493892 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.493852    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-7pmzm"]
Sep 11 19:51:49.494349 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.494101    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5bfc579bd-7pmzm" podUID="d0e62692-ccbe-44ff-bb41-c262c8e44c67" containerName="manager" containerID="cri-o://4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03" gracePeriod=30
Sep 11 19:51:49.745026 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.744957    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:51:49.797554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.797517    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-metrics-certs\") pod \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") "
Sep 11 19:51:49.797554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.797562    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rfsv\" (UniqueName: \"kubernetes.io/projected/d0e62692-ccbe-44ff-bb41-c262c8e44c67-kube-api-access-8rfsv\") pod \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") "
Sep 11 19:51:49.797783 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.797668    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-webhook-cert\") pod \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\" (UID: \"d0e62692-ccbe-44ff-bb41-c262c8e44c67\") "
Sep 11 19:51:49.799767 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.799735    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d0e62692-ccbe-44ff-bb41-c262c8e44c67" (UID: "d0e62692-ccbe-44ff-bb41-c262c8e44c67"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:51:49.799943 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.799913    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e62692-ccbe-44ff-bb41-c262c8e44c67-kube-api-access-8rfsv" (OuterVolumeSpecName: "kube-api-access-8rfsv") pod "d0e62692-ccbe-44ff-bb41-c262c8e44c67" (UID: "d0e62692-ccbe-44ff-bb41-c262c8e44c67"). InnerVolumeSpecName "kube-api-access-8rfsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:51:49.800040 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.799987    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d0e62692-ccbe-44ff-bb41-c262c8e44c67" (UID: "d0e62692-ccbe-44ff-bb41-c262c8e44c67"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:51:49.898863 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.898816    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:49.898863 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.898858    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rfsv\" (UniqueName: \"kubernetes.io/projected/d0e62692-ccbe-44ff-bb41-c262c8e44c67-kube-api-access-8rfsv\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:49.898863 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:49.898868    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0e62692-ccbe-44ff-bb41-c262c8e44c67-webhook-cert\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:51:50.393002 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:50.392965    2583 generic.go:356] "Generic (PLEG): container finished" podID="d0e62692-ccbe-44ff-bb41-c262c8e44c67" containerID="4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03" exitCode=0
Sep 11 19:51:50.393002 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:50.393008    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bfc579bd-7pmzm"
Sep 11 19:51:50.393220 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:50.393007    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bfc579bd-7pmzm" event={"ID":"d0e62692-ccbe-44ff-bb41-c262c8e44c67","Type":"ContainerDied","Data":"4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03"}
Sep 11 19:51:50.393220 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:50.393135    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bfc579bd-7pmzm" event={"ID":"d0e62692-ccbe-44ff-bb41-c262c8e44c67","Type":"ContainerDied","Data":"4c701af92352ae8cd042f1b45cee3351e48188d44fe1171c9989bc908f369a7d"}
Sep 11 19:51:50.393220 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:50.393167    2583 scope.go:117] "RemoveContainer" containerID="4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03"
Sep 11 19:51:50.403106 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:50.403090    2583 scope.go:117] "RemoveContainer" containerID="4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03"
Sep 11 19:51:50.403421 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:51:50.403397    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03\": container with ID starting with 4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03 not found: ID does not exist" containerID="4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03"
Sep 11 19:51:50.403491 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:50.403434    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03"} err="failed to get container status \"4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03\": rpc error: code = NotFound desc = could not find container \"4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03\": container with ID starting with 4dd42dcfe775fa297bb24b9128c6cab1fa3a059c3056ee060c7810d35b865a03 not found: ID does not exist"
Sep 11 19:51:50.414158 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:50.414132    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-7pmzm"]
Sep 11 19:51:50.417870 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:50.417849    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-7pmzm"]
Sep 11 19:51:50.546229 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:51:50.546193    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e62692-ccbe-44ff-bb41-c262c8e44c67" path="/var/lib/kubelet/pods/d0e62692-ccbe-44ff-bb41-c262c8e44c67/volumes"
Sep 11 19:52:07.490558 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:07.490524    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:52:23.808103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:23.808060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:52:23.908466 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:23.908424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:52:51.169642 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169549    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bfc579bd-cqxp2"]
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169885    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e62692-ccbe-44ff-bb41-c262c8e44c67" containerName="manager"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169897    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e62692-ccbe-44ff-bb41-c262c8e44c67" containerName="manager"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169909    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="283f6644-486d-4878-9545-8536d99f4ad5" containerName="maas-api"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169915    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="283f6644-486d-4878-9545-8536d99f4ad5" containerName="maas-api"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169921    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7381b624-acc3-4486-b0df-27ee204feb3e" containerName="maas-api"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169928    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7381b624-acc3-4486-b0df-27ee204feb3e" containerName="maas-api"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169937    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91580031-4554-4007-9f85-cb9457078723" containerName="payload-processing"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169942    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="91580031-4554-4007-9f85-cb9457078723" containerName="payload-processing"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169948    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06869ae7-9c43-4b03-9b32-490b86d7dba9" containerName="payload-processing"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169953    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="06869ae7-9c43-4b03-9b32-490b86d7dba9" containerName="payload-processing"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169963    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af2b69e-5830-40bf-87e6-81efaded35ad" containerName="payload-pre-processing"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169968    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af2b69e-5830-40bf-87e6-81efaded35ad" containerName="payload-pre-processing"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169975    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78378756-56dd-4304-8c00-d126ed20aeb1" containerName="maas-api"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169980    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="78378756-56dd-4304-8c00-d126ed20aeb1" containerName="maas-api"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169989    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e1024cb-256f-4317-8f29-a0ef4f0b491a" containerName="payload-pre-processing"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.169995    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e1024cb-256f-4317-8f29-a0ef4f0b491a" containerName="payload-pre-processing"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.170004    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd0288d1-15eb-4c2e-acff-b821666d171c" containerName="maas-api"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.170009    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd0288d1-15eb-4c2e-acff-b821666d171c" containerName="maas-api"
Sep 11 19:52:51.170053 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.170061    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7381b624-acc3-4486-b0df-27ee204feb3e" containerName="maas-api"
Sep 11 19:52:51.170840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.170069    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="91580031-4554-4007-9f85-cb9457078723" containerName="payload-processing"
Sep 11 19:52:51.170840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.170076    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="06869ae7-9c43-4b03-9b32-490b86d7dba9" containerName="payload-processing"
Sep 11 19:52:51.170840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.170081    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="283f6644-486d-4878-9545-8536d99f4ad5" containerName="maas-api"
Sep 11 19:52:51.170840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.170086    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e1024cb-256f-4317-8f29-a0ef4f0b491a" containerName="payload-pre-processing"
Sep 11 19:52:51.170840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.170092    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="78378756-56dd-4304-8c00-d126ed20aeb1" containerName="maas-api"
Sep 11 19:52:51.170840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.170097    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1af2b69e-5830-40bf-87e6-81efaded35ad" containerName="payload-pre-processing"
Sep 11 19:52:51.170840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.170102    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd0288d1-15eb-4c2e-acff-b821666d171c" containerName="maas-api"
Sep 11 19:52:51.170840 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.170107    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e62692-ccbe-44ff-bb41-c262c8e44c67" containerName="manager"
Sep 11 19:52:51.173122 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.173106    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:51.175957 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.175796    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pchrp\""
Sep 11 19:52:51.176106 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.176091    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 19:52:51.176518 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.176495    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 19:52:51.176892 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.176874    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 19:52:51.182988 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.182409    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-cqxp2"]
Sep 11 19:52:51.252171 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.252123    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-metrics-certs\") pod \"maas-controller-5bfc579bd-cqxp2\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") " pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:51.252389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.252215    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m9kw\" (UniqueName: \"kubernetes.io/projected/a5558f19-4623-43ac-a438-66fcdc1634cd-kube-api-access-6m9kw\") pod \"maas-controller-5bfc579bd-cqxp2\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") " pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:51.252389 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.252269    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-webhook-cert\") pod \"maas-controller-5bfc579bd-cqxp2\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") " pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:51.353046 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.353000    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m9kw\" (UniqueName: \"kubernetes.io/projected/a5558f19-4623-43ac-a438-66fcdc1634cd-kube-api-access-6m9kw\") pod \"maas-controller-5bfc579bd-cqxp2\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") " pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:51.353046 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.353052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-webhook-cert\") pod \"maas-controller-5bfc579bd-cqxp2\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") " pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:51.353269 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.353104    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-metrics-certs\") pod \"maas-controller-5bfc579bd-cqxp2\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") " pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:51.355712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.355682    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-webhook-cert\") pod \"maas-controller-5bfc579bd-cqxp2\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") " pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:51.355829 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.355719    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-metrics-certs\") pod \"maas-controller-5bfc579bd-cqxp2\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") " pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:51.360593 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.360569    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m9kw\" (UniqueName: \"kubernetes.io/projected/a5558f19-4623-43ac-a438-66fcdc1634cd-kube-api-access-6m9kw\") pod \"maas-controller-5bfc579bd-cqxp2\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") " pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:51.487621 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.487537    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:51.760806 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.760728    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-cqxp2"]
Sep 11 19:52:51.763038 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:52:51.763011    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5558f19_4623_43ac_a438_66fcdc1634cd.slice/crio-d6b869b405af1e90f95f283cce67ea3d79422fdceb84a53b29e6dd11ac66222b WatchSource:0}: Error finding container d6b869b405af1e90f95f283cce67ea3d79422fdceb84a53b29e6dd11ac66222b: Status 404 returned error can't find the container with id d6b869b405af1e90f95f283cce67ea3d79422fdceb84a53b29e6dd11ac66222b
Sep 11 19:52:51.764360 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:51.764343    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 19:52:52.677048 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:52.677011    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bfc579bd-cqxp2" event={"ID":"a5558f19-4623-43ac-a438-66fcdc1634cd","Type":"ContainerStarted","Data":"30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe"}
Sep 11 19:52:52.677481 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:52.677061    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bfc579bd-cqxp2" event={"ID":"a5558f19-4623-43ac-a438-66fcdc1634cd","Type":"ContainerStarted","Data":"d6b869b405af1e90f95f283cce67ea3d79422fdceb84a53b29e6dd11ac66222b"}
Sep 11 19:52:52.677481 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:52.677132    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:52:52.695602 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:52:52.695550    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bfc579bd-cqxp2" podStartSLOduration=1.254200437 podStartE2EDuration="1.695527915s" totalImagesPullingTime="441.327478ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:52:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:52:52.693956171 +0000 UTC m=+2076.622581081" watchObservedRunningTime="2026-09-11 19:52:52.695527915 +0000 UTC m=+2076.624152814"
Sep 11 19:53:03.684507 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:03.684471    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:53:06.615177 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:06.615141    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:53:12.864306 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:12.864268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:53:12.958914 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:12.958871    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-cqxp2"]
Sep 11 19:53:12.959146 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:12.959120    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5bfc579bd-cqxp2" podUID="a5558f19-4623-43ac-a438-66fcdc1634cd" containerName="manager" containerID="cri-o://30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe" gracePeriod=30
Sep 11 19:53:13.207570 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.207536    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:53:13.248538 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.248506    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-webhook-cert\") pod \"a5558f19-4623-43ac-a438-66fcdc1634cd\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") "
Sep 11 19:53:13.248721 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.248545    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m9kw\" (UniqueName: \"kubernetes.io/projected/a5558f19-4623-43ac-a438-66fcdc1634cd-kube-api-access-6m9kw\") pod \"a5558f19-4623-43ac-a438-66fcdc1634cd\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") "
Sep 11 19:53:13.248721 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.248621    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-metrics-certs\") pod \"a5558f19-4623-43ac-a438-66fcdc1634cd\" (UID: \"a5558f19-4623-43ac-a438-66fcdc1634cd\") "
Sep 11 19:53:13.250935 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.250897    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a5558f19-4623-43ac-a438-66fcdc1634cd" (UID: "a5558f19-4623-43ac-a438-66fcdc1634cd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:53:13.251074 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.250932    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5558f19-4623-43ac-a438-66fcdc1634cd-kube-api-access-6m9kw" (OuterVolumeSpecName: "kube-api-access-6m9kw") pod "a5558f19-4623-43ac-a438-66fcdc1634cd" (UID: "a5558f19-4623-43ac-a438-66fcdc1634cd"). InnerVolumeSpecName "kube-api-access-6m9kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:53:13.251074 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.250995    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a5558f19-4623-43ac-a438-66fcdc1634cd" (UID: "a5558f19-4623-43ac-a438-66fcdc1634cd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:53:13.349848 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.349800    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:53:13.349848 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.349856    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a5558f19-4623-43ac-a438-66fcdc1634cd-webhook-cert\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:53:13.350066 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.349867    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m9kw\" (UniqueName: \"kubernetes.io/projected/a5558f19-4623-43ac-a438-66fcdc1634cd-kube-api-access-6m9kw\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:53:13.775176 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.775141    2583 generic.go:356] "Generic (PLEG): container finished" podID="a5558f19-4623-43ac-a438-66fcdc1634cd" containerID="30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe" exitCode=0
Sep 11 19:53:13.775382 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.775184    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bfc579bd-cqxp2"
Sep 11 19:53:13.775382 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.775212    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bfc579bd-cqxp2" event={"ID":"a5558f19-4623-43ac-a438-66fcdc1634cd","Type":"ContainerDied","Data":"30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe"}
Sep 11 19:53:13.775382 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.775281    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bfc579bd-cqxp2" event={"ID":"a5558f19-4623-43ac-a438-66fcdc1634cd","Type":"ContainerDied","Data":"d6b869b405af1e90f95f283cce67ea3d79422fdceb84a53b29e6dd11ac66222b"}
Sep 11 19:53:13.775382 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.775311    2583 scope.go:117] "RemoveContainer" containerID="30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe"
Sep 11 19:53:13.784319 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.784298    2583 scope.go:117] "RemoveContainer" containerID="30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe"
Sep 11 19:53:13.784608 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:53:13.784589    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe\": container with ID starting with 30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe not found: ID does not exist" containerID="30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe"
Sep 11 19:53:13.784665 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.784619    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe"} err="failed to get container status \"30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe\": rpc error: code = NotFound desc = could not find container \"30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe\": container with ID starting with 30d6d8c9b0d992724b6e4d02dd490f47750b739b0591682315c08371589d36fe not found: ID does not exist"
Sep 11 19:53:13.796924 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.796396    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-cqxp2"]
Sep 11 19:53:13.804104 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:13.804081    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-cqxp2"]
Sep 11 19:53:14.438972 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.438937    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:53:14.546371 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.546341    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5558f19-4623-43ac-a438-66fcdc1634cd" path="/var/lib/kubelet/pods/a5558f19-4623-43ac-a438-66fcdc1634cd/volumes"
Sep 11 19:53:14.553391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.553357    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bfc579bd-k75kl"]
Sep 11 19:53:14.553720 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.553706    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5558f19-4623-43ac-a438-66fcdc1634cd" containerName="manager"
Sep 11 19:53:14.553771 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.553723    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5558f19-4623-43ac-a438-66fcdc1634cd" containerName="manager"
Sep 11 19:53:14.553818 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.553788    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5558f19-4623-43ac-a438-66fcdc1634cd" containerName="manager"
Sep 11 19:53:14.558274 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.558257    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:14.560765 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.560746    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 19:53:14.561857 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.561839    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 19:53:14.561857 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.561849    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 19:53:14.562010 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.561844    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pchrp\""
Sep 11 19:53:14.567354 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.567332    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-k75kl"]
Sep 11 19:53:14.661713 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.661676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9qbp\" (UniqueName: \"kubernetes.io/projected/aa590f0c-ff79-41e9-97cd-1939b1d2adf3-kube-api-access-z9qbp\") pod \"maas-controller-5bfc579bd-k75kl\" (UID: \"aa590f0c-ff79-41e9-97cd-1939b1d2adf3\") " pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:14.661713 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.661714    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa590f0c-ff79-41e9-97cd-1939b1d2adf3-metrics-certs\") pod \"maas-controller-5bfc579bd-k75kl\" (UID: \"aa590f0c-ff79-41e9-97cd-1939b1d2adf3\") " pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:14.661955 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.661737    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa590f0c-ff79-41e9-97cd-1939b1d2adf3-webhook-cert\") pod \"maas-controller-5bfc579bd-k75kl\" (UID: \"aa590f0c-ff79-41e9-97cd-1939b1d2adf3\") " pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:14.763180 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.763081    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9qbp\" (UniqueName: \"kubernetes.io/projected/aa590f0c-ff79-41e9-97cd-1939b1d2adf3-kube-api-access-z9qbp\") pod \"maas-controller-5bfc579bd-k75kl\" (UID: \"aa590f0c-ff79-41e9-97cd-1939b1d2adf3\") " pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:14.763180 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.763132    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa590f0c-ff79-41e9-97cd-1939b1d2adf3-metrics-certs\") pod \"maas-controller-5bfc579bd-k75kl\" (UID: \"aa590f0c-ff79-41e9-97cd-1939b1d2adf3\") " pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:14.763419 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.763277    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa590f0c-ff79-41e9-97cd-1939b1d2adf3-webhook-cert\") pod \"maas-controller-5bfc579bd-k75kl\" (UID: \"aa590f0c-ff79-41e9-97cd-1939b1d2adf3\") " pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:14.765750 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.765719    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa590f0c-ff79-41e9-97cd-1939b1d2adf3-metrics-certs\") pod \"maas-controller-5bfc579bd-k75kl\" (UID: \"aa590f0c-ff79-41e9-97cd-1939b1d2adf3\") " pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:14.765875 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.765751    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa590f0c-ff79-41e9-97cd-1939b1d2adf3-webhook-cert\") pod \"maas-controller-5bfc579bd-k75kl\" (UID: \"aa590f0c-ff79-41e9-97cd-1939b1d2adf3\") " pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:14.770982 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.770963    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9qbp\" (UniqueName: \"kubernetes.io/projected/aa590f0c-ff79-41e9-97cd-1939b1d2adf3-kube-api-access-z9qbp\") pod \"maas-controller-5bfc579bd-k75kl\" (UID: \"aa590f0c-ff79-41e9-97cd-1939b1d2adf3\") " pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:14.870079 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:14.870026    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:15.139038 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:15.139015    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bfc579bd-k75kl"]
Sep 11 19:53:15.141162 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:53:15.141133    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa590f0c_ff79_41e9_97cd_1939b1d2adf3.slice/crio-5a1e3711e9df73fdad9668e688fde13188bea847e088e3f4d277e3ce273af1d6 WatchSource:0}: Error finding container 5a1e3711e9df73fdad9668e688fde13188bea847e088e3f4d277e3ce273af1d6: Status 404 returned error can't find the container with id 5a1e3711e9df73fdad9668e688fde13188bea847e088e3f4d277e3ce273af1d6
Sep 11 19:53:15.787271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:15.787162    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bfc579bd-k75kl" event={"ID":"aa590f0c-ff79-41e9-97cd-1939b1d2adf3","Type":"ContainerStarted","Data":"b3b2bd908fca95d5a0f1cc6a10aeae1dbe779e73526c6d13af5afee21d16b395"}
Sep 11 19:53:15.787271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:15.787212    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bfc579bd-k75kl" event={"ID":"aa590f0c-ff79-41e9-97cd-1939b1d2adf3","Type":"ContainerStarted","Data":"5a1e3711e9df73fdad9668e688fde13188bea847e088e3f4d277e3ce273af1d6"}
Sep 11 19:53:15.787271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:15.787255    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:53:15.803674 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:15.803624    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bfc579bd-k75kl" podStartSLOduration=1.453409663 podStartE2EDuration="1.803607926s" totalImagesPullingTime="350.198263ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:53:15.801846434 +0000 UTC m=+2099.730471332" watchObservedRunningTime="2026-09-11 19:53:15.803607926 +0000 UTC m=+2099.732232824"
Sep 11 19:53:26.794160 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:53:26.794130    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bfc579bd-k75kl"
Sep 11 19:54:16.108953 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:54:16.108863    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:55:14.585257 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:14.585205    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:55:16.897564 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:16.897530    2583 scope.go:117] "RemoveContainer" containerID="52cb7d070274f8df697e8ed0f604f24fe81561c1a7cfa3b2f59bb8b418fa88db"
Sep 11 19:55:30.882677 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:30.882635    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"]
Sep 11 19:55:30.886187 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:30.886162    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"
Sep 11 19:55:30.889171 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:30.889146    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rddbf\""
Sep 11 19:55:30.908272 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:30.908224    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"]
Sep 11 19:55:30.978262 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:30.978207    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv4hs\" (UniqueName: \"kubernetes.io/projected/c42524d0-ed7e-4e47-89da-f82eac7904ce-kube-api-access-hv4hs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2q724\" (UID: \"c42524d0-ed7e-4e47-89da-f82eac7904ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"
Sep 11 19:55:30.978453 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:30.978357    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c42524d0-ed7e-4e47-89da-f82eac7904ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2q724\" (UID: \"c42524d0-ed7e-4e47-89da-f82eac7904ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"
Sep 11 19:55:31.079458 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:31.079414    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c42524d0-ed7e-4e47-89da-f82eac7904ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2q724\" (UID: \"c42524d0-ed7e-4e47-89da-f82eac7904ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"
Sep 11 19:55:31.079632 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:31.079470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv4hs\" (UniqueName: \"kubernetes.io/projected/c42524d0-ed7e-4e47-89da-f82eac7904ce-kube-api-access-hv4hs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2q724\" (UID: \"c42524d0-ed7e-4e47-89da-f82eac7904ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"
Sep 11 19:55:31.079806 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:31.079785    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c42524d0-ed7e-4e47-89da-f82eac7904ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2q724\" (UID: \"c42524d0-ed7e-4e47-89da-f82eac7904ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"
Sep 11 19:55:31.088336 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:31.088305    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv4hs\" (UniqueName: \"kubernetes.io/projected/c42524d0-ed7e-4e47-89da-f82eac7904ce-kube-api-access-hv4hs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2q724\" (UID: \"c42524d0-ed7e-4e47-89da-f82eac7904ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"
Sep 11 19:55:31.195815 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:31.195717    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"
Sep 11 19:55:31.475821 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:31.475795    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"]
Sep 11 19:55:31.478526 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:55:31.478494    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42524d0_ed7e_4e47_89da_f82eac7904ce.slice/crio-4754a8c754a641c01a1337749164affa84ec6f9088d67e91bc1dd52bea311186 WatchSource:0}: Error finding container 4754a8c754a641c01a1337749164affa84ec6f9088d67e91bc1dd52bea311186: Status 404 returned error can't find the container with id 4754a8c754a641c01a1337749164affa84ec6f9088d67e91bc1dd52bea311186
Sep 11 19:55:32.412450 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:32.412413    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724" event={"ID":"c42524d0-ed7e-4e47-89da-f82eac7904ce","Type":"ContainerStarted","Data":"348705cdf799cbce79faa5a256977babddb4b793e5b7124db36aa77853856913"}
Sep 11 19:55:32.412970 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:32.412471    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"
Sep 11 19:55:32.412970 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:32.412489    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724" event={"ID":"c42524d0-ed7e-4e47-89da-f82eac7904ce","Type":"ContainerStarted","Data":"4754a8c754a641c01a1337749164affa84ec6f9088d67e91bc1dd52bea311186"}
Sep 11 19:55:32.434484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:32.434426    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724" podStartSLOduration=2.434408167 podStartE2EDuration="2.434408167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:55:32.433053809 +0000 UTC m=+2236.361678716" watchObservedRunningTime="2026-09-11 19:55:32.434408167 +0000 UTC m=+2236.363033066"
Sep 11 19:55:43.419554 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:55:43.419472    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2q724"
Sep 11 19:56:19.574204 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:56:19.574161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:56:34.436214 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:56:34.436180    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:57:04.409918 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:04.409883    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:57:06.016278 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:06.016222    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:57:32.606771 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.606691    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6db8957965-7gh29"]
Sep 11 19:57:32.610651 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.610631    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.619905 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.619880    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6db8957965-7gh29"]
Sep 11 19:57:32.800169 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.800122    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef5e233-917f-4058-9828-dd7cf2a2df74-metrics-certs\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.800169 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.800166    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ef5e233-917f-4058-9828-dd7cf2a2df74-openshift-service-ca\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.800466 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.800231    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j84n8\" (UniqueName: \"kubernetes.io/projected/0ef5e233-917f-4058-9828-dd7cf2a2df74-kube-api-access-j84n8\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.800466 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.800275    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ef5e233-917f-4058-9828-dd7cf2a2df74-maas-api-tls\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.901437 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.901353    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef5e233-917f-4058-9828-dd7cf2a2df74-metrics-certs\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.901437 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.901395    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ef5e233-917f-4058-9828-dd7cf2a2df74-openshift-service-ca\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.901437 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.901426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j84n8\" (UniqueName: \"kubernetes.io/projected/0ef5e233-917f-4058-9828-dd7cf2a2df74-kube-api-access-j84n8\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.901696 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.901454    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ef5e233-917f-4058-9828-dd7cf2a2df74-maas-api-tls\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.902166 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.902147    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ef5e233-917f-4058-9828-dd7cf2a2df74-openshift-service-ca\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.904018 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.903986    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ef5e233-917f-4058-9828-dd7cf2a2df74-maas-api-tls\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.904145 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.904048    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef5e233-917f-4058-9828-dd7cf2a2df74-metrics-certs\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.909795 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.909764    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j84n8\" (UniqueName: \"kubernetes.io/projected/0ef5e233-917f-4058-9828-dd7cf2a2df74-kube-api-access-j84n8\") pod \"maas-api-6db8957965-7gh29\" (UID: \"0ef5e233-917f-4058-9828-dd7cf2a2df74\") " pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:32.922669 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:32.922642    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:33.197667 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:33.197635    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6db8957965-7gh29"]
Sep 11 19:57:33.200273 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:57:33.200220    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef5e233_917f_4058_9828_dd7cf2a2df74.slice/crio-735f043f34f9f123cd627de8e64bc40d83d731d5cd4fc7b1b086961d02c6b6a4 WatchSource:0}: Error finding container 735f043f34f9f123cd627de8e64bc40d83d731d5cd4fc7b1b086961d02c6b6a4: Status 404 returned error can't find the container with id 735f043f34f9f123cd627de8e64bc40d83d731d5cd4fc7b1b086961d02c6b6a4
Sep 11 19:57:33.983378 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:33.983343    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29" event={"ID":"0ef5e233-917f-4058-9828-dd7cf2a2df74","Type":"ContainerStarted","Data":"a09a86f139d6aff2cb7d724603c1591c29f20cb5fd582b29b4a60c2df0d55735"}
Sep 11 19:57:33.983769 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:33.983392    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29" event={"ID":"0ef5e233-917f-4058-9828-dd7cf2a2df74","Type":"ContainerStarted","Data":"735f043f34f9f123cd627de8e64bc40d83d731d5cd4fc7b1b086961d02c6b6a4"}
Sep 11 19:57:33.983769 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:33.983481    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:34.001023 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:34.000972    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29" podStartSLOduration=1.660383964 podStartE2EDuration="2.000953829s" totalImagesPullingTime="340.569865ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:57:34.000564524 +0000 UTC m=+2357.929189442" watchObservedRunningTime="2026-09-11 19:57:34.000953829 +0000 UTC m=+2357.929578748"
Sep 11 19:57:39.995110 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:39.995082    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6db8957965-7gh29"
Sep 11 19:57:40.039667 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:40.039625    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"]
Sep 11 19:57:40.039931 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:40.039903    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg" podUID="29834c3b-3aac-4dfa-b02b-8509d49ed264" containerName="maas-api" containerID="cri-o://00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9" gracePeriod=30
Sep 11 19:57:41.091205 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.091163    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg" podUID="29834c3b-3aac-4dfa-b02b-8509d49ed264" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.29:8443/health\": dial tcp 10.133.0.29:8443: connect: connection refused"
Sep 11 19:57:41.291004 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.290973    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:57:41.385075 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.385046    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-metrics-certs\") pod \"29834c3b-3aac-4dfa-b02b-8509d49ed264\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") "
Sep 11 19:57:41.385290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.385100    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29834c3b-3aac-4dfa-b02b-8509d49ed264-openshift-service-ca\") pod \"29834c3b-3aac-4dfa-b02b-8509d49ed264\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") "
Sep 11 19:57:41.385290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.385132    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m28bs\" (UniqueName: \"kubernetes.io/projected/29834c3b-3aac-4dfa-b02b-8509d49ed264-kube-api-access-m28bs\") pod \"29834c3b-3aac-4dfa-b02b-8509d49ed264\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") "
Sep 11 19:57:41.385290 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.385278    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-maas-api-tls\") pod \"29834c3b-3aac-4dfa-b02b-8509d49ed264\" (UID: \"29834c3b-3aac-4dfa-b02b-8509d49ed264\") "
Sep 11 19:57:41.385557 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.385528    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29834c3b-3aac-4dfa-b02b-8509d49ed264-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "29834c3b-3aac-4dfa-b02b-8509d49ed264" (UID: "29834c3b-3aac-4dfa-b02b-8509d49ed264"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 19:57:41.387578 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.387554    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "29834c3b-3aac-4dfa-b02b-8509d49ed264" (UID: "29834c3b-3aac-4dfa-b02b-8509d49ed264"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:57:41.387578 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.387569    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29834c3b-3aac-4dfa-b02b-8509d49ed264-kube-api-access-m28bs" (OuterVolumeSpecName: "kube-api-access-m28bs") pod "29834c3b-3aac-4dfa-b02b-8509d49ed264" (UID: "29834c3b-3aac-4dfa-b02b-8509d49ed264"). InnerVolumeSpecName "kube-api-access-m28bs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:57:41.387679 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.387589    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "29834c3b-3aac-4dfa-b02b-8509d49ed264" (UID: "29834c3b-3aac-4dfa-b02b-8509d49ed264"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:57:41.486492 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.486451    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-maas-api-tls\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:57:41.486492 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.486483    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29834c3b-3aac-4dfa-b02b-8509d49ed264-metrics-certs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:57:41.486492 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.486496    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29834c3b-3aac-4dfa-b02b-8509d49ed264-openshift-service-ca\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:57:41.486737 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:41.486508    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m28bs\" (UniqueName: \"kubernetes.io/projected/29834c3b-3aac-4dfa-b02b-8509d49ed264-kube-api-access-m28bs\") on node \"ip-10-0-135-147.ec2.internal\" DevicePath \"\""
Sep 11 19:57:42.021137 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:42.021102    2583 generic.go:356] "Generic (PLEG): container finished" podID="29834c3b-3aac-4dfa-b02b-8509d49ed264" containerID="00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9" exitCode=0
Sep 11 19:57:42.021343 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:42.021146    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"
Sep 11 19:57:42.021343 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:42.021188    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg" event={"ID":"29834c3b-3aac-4dfa-b02b-8509d49ed264","Type":"ContainerDied","Data":"00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9"}
Sep 11 19:57:42.021343 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:42.021252    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f476c4797-7xxqg" event={"ID":"29834c3b-3aac-4dfa-b02b-8509d49ed264","Type":"ContainerDied","Data":"d8d073e1bb0b67e5441790aa73b4fa048aa51338d5a87bf8835b8f03beeae1e4"}
Sep 11 19:57:42.021343 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:42.021276    2583 scope.go:117] "RemoveContainer" containerID="00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9"
Sep 11 19:57:42.030335 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:42.030316    2583 scope.go:117] "RemoveContainer" containerID="00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9"
Sep 11 19:57:42.030614 ip-10-0-135-147 kubenswrapper[2583]: E0911 19:57:42.030593    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9\": container with ID starting with 00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9 not found: ID does not exist" containerID="00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9"
Sep 11 19:57:42.030664 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:42.030622    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9"} err="failed to get container status \"00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9\": rpc error: code = NotFound desc = could not find container \"00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9\": container with ID starting with 00776d0475ac68511651e77190e8f94f89928bb04306cad1a01e0c0c56940fc9 not found: ID does not exist"
Sep 11 19:57:42.041376 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:42.041349    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"]
Sep 11 19:57:42.045179 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:42.045154    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-f476c4797-7xxqg"]
Sep 11 19:57:42.546167 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:57:42.546134    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29834c3b-3aac-4dfa-b02b-8509d49ed264" path="/var/lib/kubelet/pods/29834c3b-3aac-4dfa-b02b-8509d49ed264/volumes"
Sep 11 19:58:04.029429 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:04.029384    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bfc579bd-k75kl_aa590f0c-ff79-41e9-97cd-1939b1d2adf3/manager/0.log"
Sep 11 19:58:04.611835 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:04.611803    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6474799d6-hj8vr_603632b5-75d8-4672-8126-88b4b338c3b0/copy-manifests/0.log"
Sep 11 19:58:04.618509 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:04.618486    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6474799d6-hj8vr_603632b5-75d8-4672-8126-88b4b338c3b0/manager/0.log"
Sep 11 19:58:04.968880 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:04.968802    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6db8957965-7gh29_0ef5e233-917f-4058-9828-dd7cf2a2df74/maas-api/0.log"
Sep 11 19:58:05.398712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:05.398680    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-bxbpq_1ff29e22-22fe-48a6-b73c-14b7273312c7/postgres/0.log"
Sep 11 19:58:06.465105 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:06.465078    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-56j7m_dae641bf-fc1b-464f-ba54-c4452159b6c6/kuadrant-console-plugin/0.log"
Sep 11 19:58:06.578051 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:06.578019    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-2q724_c42524d0-ed7e-4e47-89da-f82eac7904ce/manager/0.log"
Sep 11 19:58:07.478094 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:07.478062    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57c686645c-pshmw_0f330021-82db-481e-abfc-3d37913c42bb/kube-auth-proxy/0.log"
Sep 11 19:58:07.824721 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:07.824638    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6st4w_4a7b30ce-04ec-43b8-acff-7ce0680ca302/payload-processing/0.log"
Sep 11 19:58:08.586720 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:08.586650    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-qlb4q_59565165-a6c6-4c39-b7a2-345d08c9b9a3/main/0.log"
Sep 11 19:58:08.912882 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:08.912851    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-9bvgj_40676762-202e-448a-ae0c-21b6417e0357/main/0.log"
Sep 11 19:58:14.619984 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:14.619951    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:58:14.737599 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:14.737559    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log" limit=1024
Sep 11 19:58:17.643310 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:17.643281    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-69wfq_c2c8ebda-08bc-4433-9b6d-e4344b72ee3b/global-pull-secret-syncer/0.log"
Sep 11 19:58:17.770260 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:17.770215    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cctlc_bd636fcf-c98a-4d32-a54f-10aa4648580b/konnectivity-agent/0.log"
Sep 11 19:58:17.865166 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:17.865142    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-147.ec2.internal_64cf4cd8df0a6677bea382d1c5122f9a/haproxy/0.log"
Sep 11 19:58:22.553219 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:22.553191    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-56j7m_dae641bf-fc1b-464f-ba54-c4452159b6c6/kuadrant-console-plugin/0.log"
Sep 11 19:58:22.599382 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:22.599351    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-2q724_c42524d0-ed7e-4e47-89da-f82eac7904ce/manager/0.log"
Sep 11 19:58:24.260448 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.260361    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-tf26r_b4168c7b-8b2a-44d6-a77e-896e0a65b4fb/kube-state-metrics/0.log"
Sep 11 19:58:24.281157 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.281125    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-tf26r_b4168c7b-8b2a-44d6-a77e-896e0a65b4fb/kube-rbac-proxy-main/0.log"
Sep 11 19:58:24.303751 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.303722    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-tf26r_b4168c7b-8b2a-44d6-a77e-896e0a65b4fb/kube-rbac-proxy-self/0.log"
Sep 11 19:58:24.328484 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.328464    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-69d66b8655-mnbcs_9193ded0-a7a6-4bd8-bd27-78407889a914/metrics-server/0.log"
Sep 11 19:58:24.458186 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.458163    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sx7t_dcb322b1-5921-4762-a0e7-f520adef0774/node-exporter/0.log"
Sep 11 19:58:24.476718 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.476696    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sx7t_dcb322b1-5921-4762-a0e7-f520adef0774/kube-rbac-proxy/0.log"
Sep 11 19:58:24.502223 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.502198    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sx7t_dcb322b1-5921-4762-a0e7-f520adef0774/init-textfile/0.log"
Sep 11 19:58:24.603897 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.603862    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-tbrkv_1c94c623-37c1-45d1-83fa-c2663aff05ac/kube-rbac-proxy-main/0.log"
Sep 11 19:58:24.624943 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.624913    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-tbrkv_1c94c623-37c1-45d1-83fa-c2663aff05ac/kube-rbac-proxy-self/0.log"
Sep 11 19:58:24.650522 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.650481    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-tbrkv_1c94c623-37c1-45d1-83fa-c2663aff05ac/openshift-state-metrics/0.log"
Sep 11 19:58:24.727391 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.727355    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d3431ea-1dd1-4977-876f-7a7d27ae2867/prometheus/0.log"
Sep 11 19:58:24.744739 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.744709    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d3431ea-1dd1-4977-876f-7a7d27ae2867/config-reloader/0.log"
Sep 11 19:58:24.766523 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.766491    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d3431ea-1dd1-4977-876f-7a7d27ae2867/thanos-sidecar/0.log"
Sep 11 19:58:24.791474 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.791445    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d3431ea-1dd1-4977-876f-7a7d27ae2867/kube-rbac-proxy-web/0.log"
Sep 11 19:58:24.815695 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.815673    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d3431ea-1dd1-4977-876f-7a7d27ae2867/kube-rbac-proxy/0.log"
Sep 11 19:58:24.836861 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.836842    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d3431ea-1dd1-4977-876f-7a7d27ae2867/kube-rbac-proxy-thanos/0.log"
Sep 11 19:58:24.859392 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:24.859336    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d3431ea-1dd1-4977-876f-7a7d27ae2867/init-config-reloader/0.log"
Sep 11 19:58:25.083696 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:25.083671    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6c8c8556-xzzlq_a13f0d80-5d86-4ad0-aee7-1809b1b69cce/thanos-query/0.log"
Sep 11 19:58:25.107558 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:25.107531    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6c8c8556-xzzlq_a13f0d80-5d86-4ad0-aee7-1809b1b69cce/kube-rbac-proxy-web/0.log"
Sep 11 19:58:25.132729 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:25.132666    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6c8c8556-xzzlq_a13f0d80-5d86-4ad0-aee7-1809b1b69cce/kube-rbac-proxy/0.log"
Sep 11 19:58:25.157996 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:25.157965    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6c8c8556-xzzlq_a13f0d80-5d86-4ad0-aee7-1809b1b69cce/prom-label-proxy/0.log"
Sep 11 19:58:25.185358 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:25.185305    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6c8c8556-xzzlq_a13f0d80-5d86-4ad0-aee7-1809b1b69cce/kube-rbac-proxy-rules/0.log"
Sep 11 19:58:25.208814 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:25.208782    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6c8c8556-xzzlq_a13f0d80-5d86-4ad0-aee7-1809b1b69cce/kube-rbac-proxy-metrics/0.log"
Sep 11 19:58:28.319881 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.319840    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"]
Sep 11 19:58:28.320406 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.320386    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29834c3b-3aac-4dfa-b02b-8509d49ed264" containerName="maas-api"
Sep 11 19:58:28.320488 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.320410    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="29834c3b-3aac-4dfa-b02b-8509d49ed264" containerName="maas-api"
Sep 11 19:58:28.320548 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.320526    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="29834c3b-3aac-4dfa-b02b-8509d49ed264" containerName="maas-api"
Sep 11 19:58:28.323727 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.323705    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.325876 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.325858    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4pg49\"/\"default-dockercfg-nxwjj\""
Sep 11 19:58:28.326587 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.326571    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4pg49\"/\"openshift-service-ca.crt\""
Sep 11 19:58:28.326680 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.326615    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4pg49\"/\"kube-root-ca.crt\""
Sep 11 19:58:28.333773 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.333752    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"]
Sep 11 19:58:28.409184 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.409146    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-proc\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.409362 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.409196    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmxx4\" (UniqueName: \"kubernetes.io/projected/f024431e-8b0b-4017-8668-49eb11dccd17-kube-api-access-hmxx4\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.409362 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.409226    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-lib-modules\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.409362 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.409299    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-podres\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.409362 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.409338    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-sys\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.509978 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.509946    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-lib-modules\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.509978 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.509983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-podres\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.510271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.510012    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-sys\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.510271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.510067    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-proc\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.510271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.510093    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmxx4\" (UniqueName: \"kubernetes.io/projected/f024431e-8b0b-4017-8668-49eb11dccd17-kube-api-access-hmxx4\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.510271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.510125    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-podres\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.510271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.510155    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-sys\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.510271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.510133    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-lib-modules\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.510271 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.510162    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f024431e-8b0b-4017-8668-49eb11dccd17-proc\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.518930 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.518912    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmxx4\" (UniqueName: \"kubernetes.io/projected/f024431e-8b0b-4017-8668-49eb11dccd17-kube-api-access-hmxx4\") pod \"perf-node-gather-daemonset-mcz6f\" (UID: \"f024431e-8b0b-4017-8668-49eb11dccd17\") " pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.634234 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.634193    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:28.911199 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:28.911168    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"]
Sep 11 19:58:28.913390 ip-10-0-135-147 kubenswrapper[2583]: W0911 19:58:28.913357    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf024431e_8b0b_4017_8668_49eb11dccd17.slice/crio-d452436f4834ac9e2da840a2b883e851f385b0fc34a99709b404f44d9c0cd8bb WatchSource:0}: Error finding container d452436f4834ac9e2da840a2b883e851f385b0fc34a99709b404f44d9c0cd8bb: Status 404 returned error can't find the container with id d452436f4834ac9e2da840a2b883e851f385b0fc34a99709b404f44d9c0cd8bb
Sep 11 19:58:29.237889 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:29.237794    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f" event={"ID":"f024431e-8b0b-4017-8668-49eb11dccd17","Type":"ContainerStarted","Data":"9a7ee23a9a69853a0bc9b4df769aa8cef218f957463d2e55995188a9fa2d8c54"}
Sep 11 19:58:29.237889 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:29.237863    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:29.238083 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:29.237888    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f" event={"ID":"f024431e-8b0b-4017-8668-49eb11dccd17","Type":"ContainerStarted","Data":"d452436f4834ac9e2da840a2b883e851f385b0fc34a99709b404f44d9c0cd8bb"}
Sep 11 19:58:29.252685 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:29.252635    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f" podStartSLOduration=1.2526188600000001 podStartE2EDuration="1.25261886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:58:29.251619001 +0000 UTC m=+2413.180243899" watchObservedRunningTime="2026-09-11 19:58:29.25261886 +0000 UTC m=+2413.181243757"
Sep 11 19:58:29.914903 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:29.914875    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-49ldf_7e4162f7-09dd-44d1-85dd-d1380adfadef/dns/0.log"
Sep 11 19:58:29.933712 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:29.933682    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-49ldf_7e4162f7-09dd-44d1-85dd-d1380adfadef/kube-rbac-proxy/0.log"
Sep 11 19:58:30.058820 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:30.058783    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7hsvz_3754f687-4a22-4dbe-9151-d33ccbc0afd5/dns-node-resolver/0.log"
Sep 11 19:58:30.642103 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:30.642074    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-277dr_cd7f61a5-358b-44ea-aeeb-8f7b8ca446f6/node-ca/0.log"
Sep 11 19:58:32.444222 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:32.444184    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57c686645c-pshmw_0f330021-82db-481e-abfc-3d37913c42bb/kube-auth-proxy/0.log"
Sep 11 19:58:32.547144 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:32.547113    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6st4w_4a7b30ce-04ec-43b8-acff-7ce0680ca302/payload-processing/0.log"
Sep 11 19:58:33.472813 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:33.472782    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zkswt_2d2a55d8-22f4-48cd-8112-59c9328b9e05/serve-healthcheck-canary/0.log"
Sep 11 19:58:33.938991 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:33.938965    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5jj8q_c5eb3f3d-bd85-4887-8f0a-3173892dac83/kube-rbac-proxy/0.log"
Sep 11 19:58:33.961414 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:33.961390    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5jj8q_c5eb3f3d-bd85-4887-8f0a-3173892dac83/exporter/0.log"
Sep 11 19:58:33.982027 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:33.982002    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5jj8q_c5eb3f3d-bd85-4887-8f0a-3173892dac83/extractor/0.log"
Sep 11 19:58:35.253165 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:35.253135    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4pg49/perf-node-gather-daemonset-mcz6f"
Sep 11 19:58:36.468443 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:36.468414    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bfc579bd-k75kl_aa590f0c-ff79-41e9-97cd-1939b1d2adf3/manager/0.log"
Sep 11 19:58:36.785186 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:36.785079    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6474799d6-hj8vr_603632b5-75d8-4672-8126-88b4b338c3b0/manager/0.log"
Sep 11 19:58:36.803942 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:36.803905    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6474799d6-hj8vr_603632b5-75d8-4672-8126-88b4b338c3b0/copy-manifests/0.log"
Sep 11 19:58:38.290025 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:38.289928    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6df94f489d-dspkf_4d8e0559-c51c-4843-b808-a1440ed47c68/manager/0.log"
Sep 11 19:58:38.338451 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:38.338425    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-r72jf_ded261e5-0a1d-4764-8df4-33bf5b177438/openshift-lws-operator/0.log"
Sep 11 19:58:39.402019 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:39.401927    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57c686645c-pshmw_0f330021-82db-481e-abfc-3d37913c42bb/kube-auth-proxy/0.log"
Sep 11 19:58:39.483062 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:39.483031    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6st4w_4a7b30ce-04ec-43b8-acff-7ce0680ca302/payload-processing/0.log"
Sep 11 19:58:41.904294 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:41.904262    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57c686645c-pshmw_0f330021-82db-481e-abfc-3d37913c42bb/kube-auth-proxy/0.log"
Sep 11 19:58:41.999217 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:41.999184    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6st4w_4a7b30ce-04ec-43b8-acff-7ce0680ca302/payload-processing/0.log"
Sep 11 19:58:45.423297 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:45.423228    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxfp8_cc814053-fb3d-45bb-bc8e-ea849d8f1951/kube-multus-additional-cni-plugins/0.log"
Sep 11 19:58:45.441571 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:45.441546    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxfp8_cc814053-fb3d-45bb-bc8e-ea849d8f1951/egress-router-binary-copy/0.log"
Sep 11 19:58:45.459326 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:45.459300    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxfp8_cc814053-fb3d-45bb-bc8e-ea849d8f1951/cni-plugins/0.log"
Sep 11 19:58:45.476970 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:45.476951    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxfp8_cc814053-fb3d-45bb-bc8e-ea849d8f1951/bond-cni-plugin/0.log"
Sep 11 19:58:45.496056 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:45.496011    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxfp8_cc814053-fb3d-45bb-bc8e-ea849d8f1951/routeoverride-cni/0.log"
Sep 11 19:58:45.515986 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:45.515961    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxfp8_cc814053-fb3d-45bb-bc8e-ea849d8f1951/whereabouts-cni-bincopy/0.log"
Sep 11 19:58:45.533447 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:45.533423    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxfp8_cc814053-fb3d-45bb-bc8e-ea849d8f1951/whereabouts-cni/0.log"
Sep 11 19:58:45.573055 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:45.573029    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-km429_dac09145-2ebe-48a5-b5d2-c4ab764b8b9f/kube-multus/0.log"
Sep 11 19:58:45.625225 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:45.625196    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dvg7l_c1b7247e-b7a4-44db-a5dd-4817bb0df58a/network-metrics-daemon/0.log"
Sep 11 19:58:45.640342 ip-10-0-135-147 kubenswrapper[2583]: I0911 19:58:45.640316    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dvg7l_c1b7247e-b7a4-44db-a5dd-4817bb0df58a/kube-rbac-proxy/0.log"