Sep 11 16:41:17.674259 ip-10-0-135-55 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 16:41:17.674270 ip-10-0-135-55 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 16:41:17.674277 ip-10-0-135-55 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 16:41:17.674441 ip-10-0-135-55 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 16:41:27.883238 ip-10-0-135-55 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 16:41:27.883260 ip-10-0-135-55 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c090e57082b147ad83d3635a5ea94f17 --
Sep 11 16:43:22.396733 ip-10-0-135-55 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 16:43:22.760956 ip-10-0-135-55 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:43:22.760956 ip-10-0-135-55 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 16:43:22.760956 ip-10-0-135-55 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:43:22.760956 ip-10-0-135-55 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 16:43:22.760956 ip-10-0-135-55 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:43:22.762240 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.762149    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 16:43:22.765843 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765825    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:43:22.765843 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765843    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765846    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765850    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765855    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765858    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765861    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765864    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765867    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765870    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765873    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765875    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765878    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765880    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765883    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765886    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765888    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765891    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765894    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765897    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765899    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765902    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:43:22.765911 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765905    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765907    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765910    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765913    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765917    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765922    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765925    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765928    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765931    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765933    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765936    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765939    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765941    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765944    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765947    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765949    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765952    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765956    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765959    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:43:22.766436 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765961    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765964    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765968    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765971    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765974    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765977    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765979    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765982    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765984    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765987    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765989    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765992    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765994    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765997    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.765999    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766002    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766005    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766007    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766010    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766013    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:43:22.766893 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766016    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766019    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766021    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766023    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766026    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766030    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766033    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766035    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766038    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766040    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766043    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766045    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766048    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766050    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766053    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766055    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766057    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766060    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766062    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:43:22.767398 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766065    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766067    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766070    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766072    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766075    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766077    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766080    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766082    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766086    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766089    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766091    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766094    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766096    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766099    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766101    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766104    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766108    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766112    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766114    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:43:22.767889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766803    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766809    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766812    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766816    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766818    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766822    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766825    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766828    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766831    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766833    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766836    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766839    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766842    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766844    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766847    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766850    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766852    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766855    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766858    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766860    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:43:22.768400 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766863    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766867    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766870    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766872    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766875    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766878    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766880    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766883    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766886    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766888    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766891    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766893    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766896    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766899    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766902    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766905    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766908    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766911    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766914    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:43:22.768912 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766918    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766921    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766923    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766926    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766929    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766931    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766934    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766937    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766941    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766945    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766948    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766951    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766954    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766957    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766960    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766963    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766966    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766969    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766971    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:43:22.769389 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766973    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766976    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766979    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766981    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766984    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766986    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766989    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766991    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766994    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.766997    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767000    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767003    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767005    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767008    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767011    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767014    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767017    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767021    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767024    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767026    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:43:22.769886 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767029    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767032    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767034    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767036    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767039    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767041    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767045    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767047    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767050    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767053    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767056    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767059    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767061    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767064    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767067    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767070    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767072    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767075    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767077    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767080    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:43:22.770483 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767082    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767475    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767481    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767484    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767487    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767495    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767498    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767500    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767504    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767506    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767509    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767512    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767514    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767517    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767519    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767522    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767525    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767527    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767530    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767533    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:43:22.770989 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767536    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767538    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767541    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767544    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767546    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767549    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767551    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767554    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767557    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767559    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767562    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767564    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767567    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767569    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767572    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767575    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767577    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767580    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767589    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:43:22.771493 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767591    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767594    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767596    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767599    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767602    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767604    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767606    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767612    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767614    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767617    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767619    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767622    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767625    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767627    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767630    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767633    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767636    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767639    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767641    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767644    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:43:22.771980 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767646    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767649    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767651    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767654    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767656    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767659    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767662    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767664    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767667    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767671    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767675    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767678    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767686    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767689    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767692    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767695    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767697    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767700    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767702    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:43:22.772487 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767707    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767711    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767714    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767716    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767719    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767721    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767724    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767727    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767729    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767732    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767736    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767739    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767741    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767744    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767747    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767750    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767753    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767755    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767758    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767762    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:43:22.773020 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767764    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.767773    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767852    2569 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767884    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767890    2569 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767895    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767914    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767917    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767922    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767926    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767929    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767932    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767937    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767941    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767944    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767947    2569 flags.go:64] FLAG: --cgroup-root=""
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767950    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767953    2569 flags.go:64] FLAG: --client-ca-file=""
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767956    2569 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767959    2569 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767964    2569 flags.go:64] FLAG: --cluster-domain=""
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767967    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767970    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767973    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 16:43:22.773539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767976    2569 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767980    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767983    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767987    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767990    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767993    2569 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.767996    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768000    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768003    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768006    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768010    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768013    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768016    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768019    2569 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768022    2569 flags.go:64] FLAG: --enable-server="true"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768026    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768030    2569 flags.go:64] FLAG: --event-burst="100"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768033    2569 flags.go:64] FLAG: --event-qps="50"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768037    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768039    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768044    2569 flags.go:64] FLAG: --eviction-hard=""
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768048    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768051    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768054    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768057    2569 flags.go:64] FLAG: --eviction-soft=""
Sep 11 16:43:22.774117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768060    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768063    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768065    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768068    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768071    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768074    2569 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768077    2569 flags.go:64] FLAG: --feature-gates=""
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768081    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768084    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768087    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768090    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768093    2569 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768096    2569 flags.go:64] FLAG: --help="false"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768099    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-135-55.ec2.internal"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768103    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768106    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768109    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768112    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768116    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768118    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768121    2569 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768124    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768127    2569 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 16:43:22.774794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768133    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768136    2569 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768139    2569 flags.go:64] FLAG: --kube-reserved=""
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768142    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768146    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768149    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768151    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768154    2569 flags.go:64] FLAG: --lock-file=""
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768157    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768160    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768163    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768169    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768172    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768174    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768177    2569 flags.go:64] FLAG: --logging-format="text"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768180    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768183    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768186    2569 flags.go:64] FLAG: --manifest-url=""
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768189    2569 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768193    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768196    2569 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768200    2569 flags.go:64] FLAG: --max-pods="110"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768203    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768207    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768210    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 16:43:22.775411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768213    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768217    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768220    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768223    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768230    2569 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768233    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768236    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768248    2569 flags.go:64] FLAG: --pod-cidr=""
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768257    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768264    2569 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768267    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768270    2569 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768274    2569 flags.go:64] FLAG: --port="10250"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768277    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768280    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d40e8f377d062e08"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768283    2569 flags.go:64] FLAG: --qos-reserved=""
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768286    2569 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768289    2569 flags.go:64] FLAG: --register-node="true"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768292    2569 flags.go:64] FLAG: --register-with-taints=""
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768296    2569 flags.go:64] FLAG: --registry-burst="10"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768299    2569 flags.go:64] FLAG: --registry-qps="5"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768302    2569 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768304    2569 flags.go:64] FLAG: --reserved-memory=""
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768308    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768311    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 16:43:22.776025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768314    2569 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768317    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768320    2569 flags.go:64] FLAG: --runonce="false"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768323    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768326    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768331    2569 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768334    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768337    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768340    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768343    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768346    2569 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768349    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768351    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768354    2569 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768357    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768360    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768369    2569 flags.go:64] FLAG: --system-cgroups=""
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768372    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768390    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768394    2569 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768397    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768404    2569 flags.go:64] FLAG: --tls-min-version=""
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768407    2569 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768410    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768413    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 16:43:22.776673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768417    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768419    2569 flags.go:64] FLAG: --v="2"
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768423    2569 flags.go:64] FLAG: --version="false"
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768427    2569 flags.go:64] FLAG: --vmodule=""
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768431    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.768435    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768557    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768560    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768564    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768567    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768569    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768572    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768576    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768578    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768581    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768584    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768587    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768589    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768592    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768595    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:43:22.777289 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768597    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768600    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768602    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768606    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768610    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768613    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768617    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768619    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768639    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768643    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768646    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768649    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768652    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768655    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768658    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768660    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768663    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768666    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768669    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:43:22.777812 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768673    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768677    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768680    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768683    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768686    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768689    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768692    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768694    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768698    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768702    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768705    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768708    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768711    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768714    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768716    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768719    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768722    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768724    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768727    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768731    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:43:22.778312 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768734    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768736    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768739    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768742    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768744    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768747    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768750    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768752    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768755    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768758    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768761    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768764    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768766    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768769    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768772    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768774    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768777    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768780    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768782    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768785    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:43:22.778885 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768788    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768790    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768793    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768796    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768798    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768801    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768803    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768806    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768808    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768811    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768813    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768819    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768821    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768824    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768827    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768830    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768832    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768835    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768838    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768840    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:43:22.779392 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768843    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768845    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768848    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768850    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768853    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.768856    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.769597    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.776636    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.776654    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.777318    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.777325    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777395    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777400    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777403    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:43:22.779891 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777407    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777411    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777416    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777419    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777422    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777425    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777428    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777430    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777433    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777436    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777439    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777442    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777445    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777448    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777451    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777453    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777457    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777459    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777462    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777464    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:43:22.780259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777467    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777470    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777473    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777477    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777480    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777484    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777488    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777491    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777494    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777497    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777500    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777503    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777506    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777509    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777512    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777515    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777518    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777521    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777523    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777526    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:43:22.780770 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777529    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777531    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777534    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777537    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777540    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777542    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777545    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777548    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777550    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777553    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777555    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777558    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777561    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777563    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777566    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777568    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777571    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777574    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777577    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777580    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:43:22.781259 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777582    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777585    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777588    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777591    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777594    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777596    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777599    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777601    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777603    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777606    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777609    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777611    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777614    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777616    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777619    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777621    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777624    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777626    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777629    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777632    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:43:22.781797 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777635    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777638    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777640    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777644    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777648    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777650    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777653    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777656    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777659    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777663    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777666    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777669    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777672    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777675    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777678    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:43:22.782285 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777681    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.777686    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777814    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777819    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777822    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777825    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777828    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777830    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777833    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777836    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777838    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777841    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777844    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777846    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777850    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:43:22.782760 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777854    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777857    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777860    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777863    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777866    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777869    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777872    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777875    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777877    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777880    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777883    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777886    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777890    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777893    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777896    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777899    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777901    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777905    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777909    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:43:22.783422 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777911    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777914    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777917    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777921    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777923    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777926    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777929    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777932    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777934    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777937    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777940    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777942    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777945    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777947    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777950    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777953    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777955    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777958    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777960    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777963    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:43:22.784105 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777966    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777968    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777971    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777973    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777976    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777979    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777982    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777985    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777987    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777990    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777993    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777995    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.777998    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778000    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778003    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778005    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778008    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778011    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778013    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778016    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:43:22.784716 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778018    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778021    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778023    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778026    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778028    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778031    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778034    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778036    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778039    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778041    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778044    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778046    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778049    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778052    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778055    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778057    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778060    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778063    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778066    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:43:22.785203 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778069    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778071    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778074    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778077    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778080    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778082    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778085    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:22.778087    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.778092    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.778220    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.780141    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.781030    2569 server.go:1019] "Starting client certificate rotation"
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.782565    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 16:43:22.785687 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.784109    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 16:43:22.786096 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.786079    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 16:43:22.790834 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.790818    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 16:43:22.800759 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.800741    2569 log.go:25] "Validated CRI v1 runtime API"
Sep 11 16:43:22.803329 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.803317    2569 log.go:25] "Validated CRI v1 image API"
Sep 11 16:43:22.804206 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.804187    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 16:43:22.808543 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.808525    2569 fs.go:135] Filesystem UUIDs: map[3fe91d86-416a-4c2c-929d-151ea38d54e6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d6117a30-e19b-45b9-aaf0-08f205178d41:/dev/nvme0n1p3]
Sep 11 16:43:22.808599 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.808543    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 16:43:22.814053 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.814036    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 16:43:22.814506 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.814394    2569 manager.go:216] Machine: {Timestamp:2026-09-11 16:43:22.812127983 +0000 UTC m=+0.304043868 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099529 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e926087ced640230bd3502abd1efb SystemUUID:ec2e9260-87ce-d640-230b-d3502abd1efb BootID:c090e570-82b1-47ad-83d3-635a5ea94f17 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:81:d6:3f:29:9f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:81:d6:3f:29:9f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:c9:e0:c3:5e:94 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 16:43:22.814506 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.814505    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 16:43:22.814602 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.814590    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 16:43:22.817519 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.817486    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 16:43:22.817658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.817521    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-55.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 16:43:22.818223 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.818213    2569 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 16:43:22.818258 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.818225    2569 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 16:43:22.818258 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.818234    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 16:43:22.818258 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.818248    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 16:43:22.819213 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.819202    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 16:43:22.822348 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.822335    2569 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 16:43:22.822485 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.822475    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 16:43:22.822527 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.822506    2569 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 16:43:22.822527 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.822516    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 16:43:22.822628 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.822531    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 16:43:22.822628 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.822541    2569 kubelet.go:398] "Adding apiserver pod source"
Sep 11 16:43:22.822628 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.822550    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 16:43:22.822946 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.822932    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 16:43:22.823002 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.822951    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 16:43:22.825084 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.825065    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 16:43:22.827813 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.827789    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 16:43:22.827889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.827834    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 16:43:22.830251 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830027    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 16:43:22.830338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830264    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 16:43:22.830338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830273    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 16:43:22.830338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830279    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 16:43:22.830338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830288    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 16:43:22.830338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830294    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 16:43:22.830338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830308    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 16:43:22.830338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830313    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 16:43:22.830338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830320    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 16:43:22.830338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830326    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 16:43:22.830338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830334    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 16:43:22.830338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830339    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 16:43:22.831009 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.830998    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 16:43:22.831009 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.831010    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 16:43:22.834787 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.834770    2569 server.go:1268] "Started kubelet"
Sep 11 16:43:22.834893 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.834837    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 16:43:22.834893 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.834852    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 16:43:22.835047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.834927    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 16:43:22.835849 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.835833    2569 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 16:43:22.835857 ip-10-0-135-55 systemd[1]: Started Kubernetes Kubelet.
Sep 11 16:43:22.837061 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.837043    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 16:43:22.839919 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.839715    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-55.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 16:43:22.840757 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.840715    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 16:43:22.841533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.841515    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 16:43:22.842230 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.842216    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 16:43:22.842409 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.842367    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:22.842827 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.842810    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 16:43:22.842945 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.842829    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 16:43:22.842945 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.842845    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 16:43:22.843030 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.842958    2569 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 16:43:22.843030 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.842968    2569 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 16:43:22.843126 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.843032    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 16:43:22.843126 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.843048    2569 factory.go:55] Registering systemd factory
Sep 11 16:43:22.843126 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.843056    2569 factory.go:223] Registration of the systemd container factory successfully
Sep 11 16:43:22.843268 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.843186    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 16:43:22.843268 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.843237    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 16:43:22.843400 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.843278    2569 factory.go:153] Registering CRI-O factory
Sep 11 16:43:22.843400 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.843291    2569 factory.go:223] Registration of the crio container factory successfully
Sep 11 16:43:22.843400 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.843308    2569 factory.go:103] Registering Raw factory
Sep 11 16:43:22.843400 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.843321    2569 manager.go:1196] Started watching for new ooms in manager
Sep 11 16:43:22.843767 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.843754    2569 manager.go:349] Starting recovery of all containers
Sep 11 16:43:22.844398 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.843407    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-55.ec2.internal.18d451e699a13d61  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-55.ec2.internal,UID:ip-10-0-135-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-55.ec2.internal,},FirstTimestamp:2026-09-11 16:43:22.834738529 +0000 UTC m=+0.326654405,LastTimestamp:2026-09-11 16:43:22.834738529 +0000 UTC m=+0.326654405,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-55.ec2.internal,}"
Sep 11 16:43:22.850241 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.850169    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 16:43:22.851389 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.851353    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 16:43:22.851500 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.851421    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 16:43:22.854573 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.854551    2569 manager.go:354] Recovery completed
Sep 11 16:43:22.856428 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.856406    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xwx2f"
Sep 11 16:43:22.861259 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.861241    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:43:22.863172 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.863154    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:43:22.863245 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.863184    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:43:22.863245 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.863194    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:43:22.863785 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.863771    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 16:43:22.863785 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.863784    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 16:43:22.863926 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.863793    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:43:22.863926 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.863802    2569 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 16:43:22.864472 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.864454    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:43:22.864565 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.864490    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:43:22.864565 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.864504    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:43:22.865111 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.864963    2569 policy_none.go:49] "None policy: Start"
Sep 11 16:43:22.865111 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.865113    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 16:43:22.865237 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.865125    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 16:43:22.866179 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.866168    2569 policy_none.go:47] "Start"
Sep 11 16:43:22.869048 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.868953    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-55.ec2.internal.18d451e69b5315ea  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-55.ec2.internal,UID:ip-10-0-135-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-55.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-55.ec2.internal,},FirstTimestamp:2026-09-11 16:43:22.86317105 +0000 UTC m=+0.355086925,LastTimestamp:2026-09-11 16:43:22.86317105 +0000 UTC m=+0.355086925,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-55.ec2.internal,}"
Sep 11 16:43:22.871427 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.871323    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-55.ec2.internal.18d451e69b535ad2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-55.ec2.internal,UID:ip-10-0-135-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-55.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-55.ec2.internal,},FirstTimestamp:2026-09-11 16:43:22.86318869 +0000 UTC m=+0.355104565,LastTimestamp:2026-09-11 16:43:22.86318869 +0000 UTC m=+0.355104565,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-55.ec2.internal,}"
Sep 11 16:43:22.875319 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.875249    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-55.ec2.internal.18d451e69b537e86  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-55.ec2.internal,UID:ip-10-0-135-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-55.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-55.ec2.internal,},FirstTimestamp:2026-09-11 16:43:22.86319783 +0000 UTC m=+0.355113705,LastTimestamp:2026-09-11 16:43:22.86319783 +0000 UTC m=+0.355113705,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-55.ec2.internal,}"
Sep 11 16:43:22.875942 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.875921    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xwx2f"
Sep 11 16:43:22.878858 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.878762    2569 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-55.ec2.internal.18d451e69b5315ea\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-55.ec2.internal.18d451e69b5315ea  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-55.ec2.internal,UID:ip-10-0-135-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-55.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-55.ec2.internal,},FirstTimestamp:2026-09-11 16:43:22.86317105 +0000 UTC m=+0.355086925,LastTimestamp:2026-09-11 16:43:22.864471902 +0000 UTC m=+0.356387777,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-55.ec2.internal,}"
Sep 11 16:43:22.905044 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.905025    2569 manager.go:337] "Starting Device Plugin manager"
Sep 11 16:43:22.905164 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.905071    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 16:43:22.905164 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.905085    2569 server.go:87] "Starting device plugin registration server"
Sep 11 16:43:22.905552 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.905538    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 16:43:22.905605 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.905555    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 16:43:22.905689 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.905674    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 16:43:22.905770 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.905753    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 16:43:22.905770 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.905766    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 16:43:22.906556 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.906519    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 16:43:22.906657 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:22.906571    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:22.921641 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:22.921606    2569 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2601 into the system container \"/system.slice\": <nil>"
Sep 11 16:43:23.006099 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.006063    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:43:23.007208 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.007189    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:43:23.007325 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.007226    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:43:23.007325 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.007240    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:43:23.007325 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.007274    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.020463 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.020444    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.020571 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:23.020466    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-55.ec2.internal\": node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:23.058045 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:23.058026    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:23.113682 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.113660    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 16:43:23.113803 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.113709    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 16:43:23.113803 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.113731    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 16:43:23.113883 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:23.113827    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 16:43:23.116273 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.116249    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:43:23.158444 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:23.158406    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:23.214599 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.214566    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"]
Sep 11 16:43:23.214684 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.214653    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:43:23.215671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.215656    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:43:23.215747 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.215687    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:43:23.215747 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.215700    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:43:23.217166 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.217153    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:43:23.217304 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.217279    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.217418 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.217310    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:43:23.217908 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.217891    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:43:23.217971 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.217921    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:43:23.217971 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.217932    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:43:23.217971 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.217965    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:43:23.218078 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.217988    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:43:23.218078 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.217998    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:43:23.219159 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.219145    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.219213 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.219170    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:43:23.219886 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.219860    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:43:23.219886 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.219884    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:43:23.220020 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.219896    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:43:23.243180 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:23.243160    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-55.ec2.internal\" not found" node="ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.245134 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.245112    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f16c66ff742628bc672161cb5b05510a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"f16c66ff742628bc672161cb5b05510a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.245235 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.245146    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f16c66ff742628bc672161cb5b05510a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"f16c66ff742628bc672161cb5b05510a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.245235 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.245169    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/93e5adb587d6c9f5d90710b23c25b76f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-55.ec2.internal\" (UID: \"93e5adb587d6c9f5d90710b23c25b76f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.247629 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:23.247615    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-55.ec2.internal\" not found" node="ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.258685 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:23.258666    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:23.346316 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.346281    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/93e5adb587d6c9f5d90710b23c25b76f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-55.ec2.internal\" (UID: \"93e5adb587d6c9f5d90710b23c25b76f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.346316 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.346316    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f16c66ff742628bc672161cb5b05510a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"f16c66ff742628bc672161cb5b05510a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.346529 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.346340    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f16c66ff742628bc672161cb5b05510a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"f16c66ff742628bc672161cb5b05510a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.346529 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.346418    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f16c66ff742628bc672161cb5b05510a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"f16c66ff742628bc672161cb5b05510a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.346529 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.346433    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f16c66ff742628bc672161cb5b05510a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"f16c66ff742628bc672161cb5b05510a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:23.346529 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:23.346422    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/93e5adb587d6c9f5d90710b23c25b76f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-55.ec2.internal\" (UID: \"93e5adb587d6c9f5d90710b23c25b76f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:24.444127 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:24.444099    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:24.544678 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:24.544636    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:24.630044 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:24.630014    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:24.634770 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:24.634749    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:24.645308 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:24.645288    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:24.741108 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:24.741011    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:43:24.745725 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:24.745693    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:24.846445 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:24.846413    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:24.865611 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:24.865586    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 16:43:24.865756 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:24.865736    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 16:43:24.865803 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:24.865736    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 16:43:24.947292 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:24.947260    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Sep 11 16:43:24.964455 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:24.964417    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 16:38:23 +0000 UTC" deadline="2028-03-26 07:32:56.821008965 +0000 UTC"
Sep 11 16:43:24.964455 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:24.964449    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13478h49m31.856563004s"
Sep 11 16:43:25.004526 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.004448    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:43:25.027436 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.027397    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:25.038764 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.038738    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 16:43:25.039701 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.039688    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Sep 11 16:43:25.048920 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.048901    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 16:43:25.227827 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:25.227793    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16c66ff742628bc672161cb5b05510a.slice/crio-f75107a971f8426edca99a7f744b20800dec2b89e9cd7dedcdb7995495b303bf WatchSource:0}: Error finding container f75107a971f8426edca99a7f744b20800dec2b89e9cd7dedcdb7995495b303bf: Status 404 returned error can't find the container with id f75107a971f8426edca99a7f744b20800dec2b89e9cd7dedcdb7995495b303bf
Sep 11 16:43:25.228553 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:25.228529    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e5adb587d6c9f5d90710b23c25b76f.slice/crio-29230d4e55433833501f16a922eff51ca8a4f64d71ff66e4dde8576343043a33 WatchSource:0}: Error finding container 29230d4e55433833501f16a922eff51ca8a4f64d71ff66e4dde8576343043a33: Status 404 returned error can't find the container with id 29230d4e55433833501f16a922eff51ca8a4f64d71ff66e4dde8576343043a33
Sep 11 16:43:25.233832 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.233817    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 16:43:25.496023 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.495992    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:43:25.760622 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.760544    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:43:25.908017 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.907982    2569 apiserver.go:52] "Watching apiserver"
Sep 11 16:43:25.914935 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.914910    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 16:43:25.915360 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.915336    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-b6fm6","kube-system/konnectivity-agent-jpzqw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw","openshift-cluster-node-tuning-operator/tuned-8dzr4","openshift-image-registry/node-ca-l28sb","openshift-multus/multus-additional-cni-plugins-6ngsr","openshift-multus/multus-vtmr5","openshift-multus/network-metrics-daemon-xnn4g","kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal","openshift-dns/node-resolver-gd8hl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal","openshift-network-diagnostics/network-check-target-7p27r","openshift-network-operator/iptables-alerter-pnmkk","openshift-ovn-kubernetes/ovnkube-node-dl5sh"]
Sep 11 16:43:25.918030 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.918005    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.920773 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.920186    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 16:43:25.920773 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.920272    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kcjgg\""
Sep 11 16:43:25.920773 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.920281    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 16:43:25.920773 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.920422    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 16:43:25.920773 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.920519    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 16:43:25.920773 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.920678    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pnmkk"
Sep 11 16:43:25.922823 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.922801    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 16:43:25.922933 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.922916    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:43:25.923103 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.923087    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2hfch\""
Sep 11 16:43:25.923173 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.923154    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:25.923173 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.923099    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 16:43:25.924793 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.924772    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 16:43:25.924887 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.924776    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 16:43:25.924887 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.924835    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xbj4f\""
Sep 11 16:43:25.925147 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.925127    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 16:43:25.925483 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.925370    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.927205 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.927187    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:43:25.927398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.927385    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 16:43:25.927459 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.927436    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 16:43:25.927663 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.927648    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ccjpz\""
Sep 11 16:43:25.928153 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.928001    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l28sb"
Sep 11 16:43:25.929748 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.929727    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 16:43:25.929839 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.929752    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 16:43:25.929839 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.929756    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 16:43:25.930024 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.930008    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7rvnc\""
Sep 11 16:43:25.930407 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.930388    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:25.932610 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.932592    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8kgpz\""
Sep 11 16:43:25.932746 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.932722    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 16:43:25.932904 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.932842    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:25.932994 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.932962    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 16:43:25.935535 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.935303    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:25.935535 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.935387    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 16:43:25.935535 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:25.935385    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:25.935718 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.935627    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 16:43:25.935718 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.935706    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 16:43:25.935811 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.935718    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zb52f\""
Sep 11 16:43:25.935811 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.935630    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 16:43:25.935904 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.935734    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 16:43:25.936140 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.936124    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 16:43:25.937585 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.937563    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:25.939578 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.939544    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kwmz6\""
Sep 11 16:43:25.939800 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.939767    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-etc-kubernetes\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.939879 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.939841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d58f1c39-4e28-4ee8-a405-99102811ad67-host-slash\") pod \"iptables-alerter-pnmkk\" (UID: \"d58f1c39-4e28-4ee8-a405-99102811ad67\") " pod="openshift-network-operator/iptables-alerter-pnmkk"
Sep 11 16:43:25.939972 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.939939    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-modprobe-d\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.940039 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.940019    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 16:43:25.940127 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.939991    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-sysctl-d\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.940454 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.940430    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-run-netns\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.940522 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.940490    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-var-lib-kubelet\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.940634 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.940602    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d58f1c39-4e28-4ee8-a405-99102811ad67-iptables-alerter-script\") pod \"iptables-alerter-pnmkk\" (UID: \"d58f1c39-4e28-4ee8-a405-99102811ad67\") " pod="openshift-network-operator/iptables-alerter-pnmkk"
Sep 11 16:43:25.940698 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.940659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-device-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:25.940819 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.940801    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-systemd\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.940881 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.940833    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-var-lib-kubelet\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.940881 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.940873    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnjl4\" (UniqueName: \"kubernetes.io/projected/b3c0abb4-7329-4793-9028-2090ccf7fa91-kube-api-access-hnjl4\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.941048 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.940903    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-cni-dir\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.941134 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.941099    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-run-multus-certs\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.941182 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.941140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:25.942154 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.942041    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-run\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.942236 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.942209    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-system-cni-dir\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.942286 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.942239    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-conf-dir\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.942340 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.942268    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-sysconfig\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.942422 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.942380    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-sys\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.942484 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.942417    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-lib-modules\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.942613 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.942588    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-tuned\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.942669 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.942627    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp699\" (UniqueName: \"kubernetes.io/projected/7fa5fa96-b9a1-4392-8189-3944be73baba-kube-api-access-bp699\") pod \"node-ca-l28sb\" (UID: \"7fa5fa96-b9a1-4392-8189-3944be73baba\") " pod="openshift-image-registry/node-ca-l28sb"
Sep 11 16:43:25.942669 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.942653    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjqn8\" (UniqueName: \"kubernetes.io/projected/d58f1c39-4e28-4ee8-a405-99102811ad67-kube-api-access-mjqn8\") pod \"iptables-alerter-pnmkk\" (UID: \"d58f1c39-4e28-4ee8-a405-99102811ad67\") " pod="openshift-network-operator/iptables-alerter-pnmkk"
Sep 11 16:43:25.942763 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.942687    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-registration-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:25.942763 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.942737    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-socket-dir-parent\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.943828 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.943207    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gd8hl"
Sep 11 16:43:25.943828 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.943514    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-var-lib-cni-bin\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.943828 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.943693    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-var-lib-cni-multus\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.943828 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.943726    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-daemon-config\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.944067 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.943808    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3c0abb4-7329-4793-9028-2090ccf7fa91-tmp\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.944067 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.943904    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18ea8e25-6023-41a7-8643-9f093ffeab1e-cni-binary-copy\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.944067 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.943923    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 16:43:25.944491 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.944469    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 16:43:25.944575 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.943939    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7rtv\" (UniqueName: \"kubernetes.io/projected/18ea8e25-6023-41a7-8643-9f093ffeab1e-kube-api-access-n7rtv\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.944625 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.944608    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-sysctl-conf\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.944721 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.944677    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-host\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.944787 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.944746    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fa5fa96-b9a1-4392-8189-3944be73baba-host\") pod \"node-ca-l28sb\" (UID: \"7fa5fa96-b9a1-4392-8189-3944be73baba\") " pod="openshift-image-registry/node-ca-l28sb"
Sep 11 16:43:25.944839 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.944819    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fa5fa96-b9a1-4392-8189-3944be73baba-serviceca\") pod \"node-ca-l28sb\" (UID: \"7fa5fa96-b9a1-4392-8189-3944be73baba\") " pod="openshift-image-registry/node-ca-l28sb"
Sep 11 16:43:25.944887 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.944857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-cnibin\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.944944 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.944892    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-run-k8s-cni-cncf-io\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.944996 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.944945    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-hostroot\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:25.945043 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.945006    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68x5f\" (UniqueName: \"kubernetes.io/projected/b6a311db-d6a1-4f9d-8140-b22458cf7b92-kube-api-access-68x5f\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:25.945141 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.945119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:25.945232 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.945212    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-kubernetes\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:25.945297 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.945271    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:25.945348 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:25.945306    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:25.945414 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.945356    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-socket-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:25.945478 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.945459    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-sys-fs\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:25.946134 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.946113    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k5msh\""
Sep 11 16:43:25.946311 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.946274    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 16:43:25.947209 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.947191    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 16:43:25.949165 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.949144    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:25.949249 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:25.949228    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:25.959018 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.958995    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8tp6s"
Sep 11 16:43:25.970142 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:25.970121    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8tp6s"
Sep 11 16:43:26.029375 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.029303    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 16:43:26.045856 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.045819    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-564sd\" (UniqueName: \"kubernetes.io/projected/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-kube-api-access-564sd\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.045865    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7rtv\" (UniqueName: \"kubernetes.io/projected/18ea8e25-6023-41a7-8643-9f093ffeab1e-kube-api-access-n7rtv\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.046026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.045902    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-sysctl-conf\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.046026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.045926    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-host\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.046026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.045964    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fa5fa96-b9a1-4392-8189-3944be73baba-serviceca\") pod \"node-ca-l28sb\" (UID: \"7fa5fa96-b9a1-4392-8189-3944be73baba\") " pod="openshift-image-registry/node-ca-l28sb"
Sep 11 16:43:26.046026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.045992    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-systemd-units\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046016    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-run-ovn\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046035    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-host\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046038    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-node-log\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046105    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046109    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-sysctl-conf\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-kubelet\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046157    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-ovnkube-config\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046204    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046237    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-hostroot\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046262    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68x5f\" (UniqueName: \"kubernetes.io/projected/b6a311db-d6a1-4f9d-8140-b22458cf7b92-kube-api-access-68x5f\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046287    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-kubernetes\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.046320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046312    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-run-ovn-kubernetes\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046335    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-ovn-node-metrics-cert\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046359    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-ovnkube-script-lib\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-socket-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046418    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d58f1c39-4e28-4ee8-a405-99102811ad67-host-slash\") pod \"iptables-alerter-pnmkk\" (UID: \"d58f1c39-4e28-4ee8-a405-99102811ad67\") " pod="openshift-network-operator/iptables-alerter-pnmkk"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046435    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-hostroot\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046444    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-var-lib-kubelet\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046470    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-var-lib-openvswitch\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046507    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/582ece86-42d2-4803-853b-611a440d509b-hosts-file\") pod \"node-resolver-gd8hl\" (UID: \"582ece86-42d2-4803-853b-611a440d509b\") " pod="openshift-dns/node-resolver-gd8hl"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046535    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-run-netns\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046559    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-var-lib-kubelet\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046582    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4604c98e-255c-4fa1-9266-08827d0e3761-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046607    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzf24\" (UniqueName: \"kubernetes.io/projected/4604c98e-255c-4fa1-9266-08827d0e3761-kube-api-access-hzf24\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046631    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-run-netns\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046653    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-log-socket\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5e339d2-e61e-4cf6-8f35-aafb644b4136-kubelet-config\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:26.046708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046676    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fa5fa96-b9a1-4392-8189-3944be73baba-serviceca\") pod \"node-ca-l28sb\" (UID: \"7fa5fa96-b9a1-4392-8189-3944be73baba\") " pod="openshift-image-registry/node-ca-l28sb"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046714    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-cni-dir\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046758    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-run\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046780    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-cni-dir\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046790    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-slash\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046826    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-env-overrides\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046850    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f26a8019-80a6-4f41-b39c-8417143279b8-konnectivity-ca\") pod \"konnectivity-agent-jpzqw\" (UID: \"f26a8019-80a6-4f41-b39c-8417143279b8\") " pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046877    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpmzc\" (UniqueName: \"kubernetes.io/projected/ef8b79f7-feb0-46de-898c-1db3ebd142db-kube-api-access-dpmzc\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046906    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-system-cni-dir\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046923    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-conf-dir\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046939    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-sysconfig\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.046996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-lib-modules\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047019    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-tuned\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047025    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-run-netns\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047043    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp699\" (UniqueName: \"kubernetes.io/projected/7fa5fa96-b9a1-4392-8189-3944be73baba-kube-api-access-bp699\") pod \"node-ca-l28sb\" (UID: \"7fa5fa96-b9a1-4392-8189-3944be73baba\") " pod="openshift-image-registry/node-ca-l28sb"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047063    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-run-systemd\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047063    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-kubernetes\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.047301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047066    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-var-lib-kubelet\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047095    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-cni-bin\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047151    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-run\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047163    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-var-lib-kubelet\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047201    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d58f1c39-4e28-4ee8-a405-99102811ad67-host-slash\") pod \"iptables-alerter-pnmkk\" (UID: \"d58f1c39-4e28-4ee8-a405-99102811ad67\") " pod="openshift-network-operator/iptables-alerter-pnmkk"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047253    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fa5fa96-b9a1-4392-8189-3944be73baba-host\") pod \"node-ca-l28sb\" (UID: \"7fa5fa96-b9a1-4392-8189-3944be73baba\") " pod="openshift-image-registry/node-ca-l28sb"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047254    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-conf-dir\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047300    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-socket-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047312    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjqn8\" (UniqueName: \"kubernetes.io/projected/d58f1c39-4e28-4ee8-a405-99102811ad67-kube-api-access-mjqn8\") pod \"iptables-alerter-pnmkk\" (UID: \"d58f1c39-4e28-4ee8-a405-99102811ad67\") " pod="openshift-network-operator/iptables-alerter-pnmkk"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047323    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-sysconfig\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047394    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-system-cni-dir\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047537    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fa5fa96-b9a1-4392-8189-3944be73baba-host\") pod \"node-ca-l28sb\" (UID: \"7fa5fa96-b9a1-4392-8189-3944be73baba\") " pod="openshift-image-registry/node-ca-l28sb"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047569    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-registration-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047596    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-socket-dir-parent\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047600    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-var-lib-cni-bin\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047642    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-registration-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047655    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-var-lib-cni-bin\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.047990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047657    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-lib-modules\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047667    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-socket-dir-parent\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047705    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-daemon-config\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047729    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3c0abb4-7329-4793-9028-2090ccf7fa91-tmp\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047755    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6d4q\" (UniqueName: \"kubernetes.io/projected/582ece86-42d2-4803-853b-611a440d509b-kube-api-access-s6d4q\") pod \"node-resolver-gd8hl\" (UID: \"582ece86-42d2-4803-853b-611a440d509b\") " pod="openshift-dns/node-resolver-gd8hl"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047781    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18ea8e25-6023-41a7-8643-9f093ffeab1e-cni-binary-copy\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047804    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-etc-openvswitch\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047844    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-run-openvswitch\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047866    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-cnibin\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047888    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-run-k8s-cni-cncf-io\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047911    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4604c98e-255c-4fa1-9266-08827d0e3761-cnibin\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047936    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5e339d2-e61e-4cf6-8f35-aafb644b4136-dbus\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.047981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f26a8019-80a6-4f41-b39c-8417143279b8-agent-certs\") pod \"konnectivity-agent-jpzqw\" (UID: \"f26a8019-80a6-4f41-b39c-8417143279b8\") " pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048027    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-cnibin\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048026    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582ece86-42d2-4803-853b-611a440d509b-tmp-dir\") pod \"node-resolver-gd8hl\" (UID: \"582ece86-42d2-4803-853b-611a440d509b\") " pod="openshift-dns/node-resolver-gd8hl"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048101    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-run-k8s-cni-cncf-io\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048144    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.048876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048103    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048172    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-sys-fs\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048192    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-etc-kubernetes\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048216    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-modprobe-d\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048235    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-sysctl-d\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048253    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-etc-kubernetes\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048295    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-sys-fs\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048328    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-modprobe-d\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d58f1c39-4e28-4ee8-a405-99102811ad67-iptables-alerter-script\") pod \"iptables-alerter-pnmkk\" (UID: \"d58f1c39-4e28-4ee8-a405-99102811ad67\") " pod="openshift-network-operator/iptables-alerter-pnmkk"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048387    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-device-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048413    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-systemd\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048433    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18ea8e25-6023-41a7-8643-9f093ffeab1e-multus-daemon-config\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048448    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-device-dir\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048438    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnjl4\" (UniqueName: \"kubernetes.io/projected/b3c0abb4-7329-4793-9028-2090ccf7fa91-kube-api-access-hnjl4\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048481    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18ea8e25-6023-41a7-8643-9f093ffeab1e-cni-binary-copy\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048496    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-systemd\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048413    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-sysctl-d\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.049620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048490    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4604c98e-255c-4fa1-9266-08827d0e3761-cni-binary-copy\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048544    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4604c98e-255c-4fa1-9266-08827d0e3761-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048576    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-run-multus-certs\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048603    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048614    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-run-multus-certs\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048628    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k257s\" (UniqueName: \"kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s\") pod \"network-check-target-7p27r\" (UID: \"0c446eb1-a854-404f-9106-98d00a6de58c\") " pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048656    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-sys\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048673    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b6a311db-d6a1-4f9d-8140-b22458cf7b92-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048702    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4604c98e-255c-4fa1-9266-08827d0e3761-system-cni-dir\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048732    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3c0abb4-7329-4793-9028-2090ccf7fa91-sys\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048767    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-cni-netd\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048790    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d58f1c39-4e28-4ee8-a405-99102811ad67-iptables-alerter-script\") pod \"iptables-alerter-pnmkk\" (UID: \"d58f1c39-4e28-4ee8-a405-99102811ad67\") " pod="openshift-network-operator/iptables-alerter-pnmkk"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048795    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4604c98e-255c-4fa1-9266-08827d0e3761-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048844    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-var-lib-cni-multus\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.050492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.048922    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18ea8e25-6023-41a7-8643-9f093ffeab1e-host-var-lib-cni-multus\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.051958 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.051774    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3c0abb4-7329-4793-9028-2090ccf7fa91-etc-tuned\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.051958 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.051926    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3c0abb4-7329-4793-9028-2090ccf7fa91-tmp\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.058161 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.057731    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7rtv\" (UniqueName: \"kubernetes.io/projected/18ea8e25-6023-41a7-8643-9f093ffeab1e-kube-api-access-n7rtv\") pod \"multus-vtmr5\" (UID: \"18ea8e25-6023-41a7-8643-9f093ffeab1e\") " pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.058161 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.058100    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnjl4\" (UniqueName: \"kubernetes.io/projected/b3c0abb4-7329-4793-9028-2090ccf7fa91-kube-api-access-hnjl4\") pod \"tuned-8dzr4\" (UID: \"b3c0abb4-7329-4793-9028-2090ccf7fa91\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.058365 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.058337    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp699\" (UniqueName: \"kubernetes.io/projected/7fa5fa96-b9a1-4392-8189-3944be73baba-kube-api-access-bp699\") pod \"node-ca-l28sb\" (UID: \"7fa5fa96-b9a1-4392-8189-3944be73baba\") " pod="openshift-image-registry/node-ca-l28sb"
Sep 11 16:43:26.058489 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.058402    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68x5f\" (UniqueName: \"kubernetes.io/projected/b6a311db-d6a1-4f9d-8140-b22458cf7b92-kube-api-access-68x5f\") pod \"aws-ebs-csi-driver-node-vfrvw\" (UID: \"b6a311db-d6a1-4f9d-8140-b22458cf7b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.058951 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.058928    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjqn8\" (UniqueName: \"kubernetes.io/projected/d58f1c39-4e28-4ee8-a405-99102811ad67-kube-api-access-mjqn8\") pod \"iptables-alerter-pnmkk\" (UID: \"d58f1c39-4e28-4ee8-a405-99102811ad67\") " pod="openshift-network-operator/iptables-alerter-pnmkk"
Sep 11 16:43:26.149537 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149497    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4604c98e-255c-4fa1-9266-08827d0e3761-cnibin\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.149713 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149543    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5e339d2-e61e-4cf6-8f35-aafb644b4136-dbus\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:26.149713 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149571    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f26a8019-80a6-4f41-b39c-8417143279b8-agent-certs\") pod \"konnectivity-agent-jpzqw\" (UID: \"f26a8019-80a6-4f41-b39c-8417143279b8\") " pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:26.149713 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149590    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582ece86-42d2-4803-853b-611a440d509b-tmp-dir\") pod \"node-resolver-gd8hl\" (UID: \"582ece86-42d2-4803-853b-611a440d509b\") " pod="openshift-dns/node-resolver-gd8hl"
Sep 11 16:43:26.149713 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4604c98e-255c-4fa1-9266-08827d0e3761-cni-binary-copy\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.149713 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149609    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4604c98e-255c-4fa1-9266-08827d0e3761-cnibin\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.149713 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149628    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4604c98e-255c-4fa1-9266-08827d0e3761-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.149713 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149657    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k257s\" (UniqueName: \"kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s\") pod \"network-check-target-7p27r\" (UID: \"0c446eb1-a854-404f-9106-98d00a6de58c\") " pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:26.149713 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149677    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4604c98e-255c-4fa1-9266-08827d0e3761-system-cni-dir\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.149713 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149697    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-cni-netd\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149721    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4604c98e-255c-4fa1-9266-08827d0e3761-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149747    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-564sd\" (UniqueName: \"kubernetes.io/projected/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-kube-api-access-564sd\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-systemd-units\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149792    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4604c98e-255c-4fa1-9266-08827d0e3761-system-cni-dir\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149796    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-run-ovn\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149833    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-run-ovn\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149844    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-node-log\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149875    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-cni-netd\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149873    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149906    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-kubelet\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149911    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149931    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-ovnkube-config\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149943    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-node-log\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149958    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149984    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.149985    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-systemd-units\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.150229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150042    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582ece86-42d2-4803-853b-611a440d509b-tmp-dir\") pod \"node-resolver-gd8hl\" (UID: \"582ece86-42d2-4803-853b-611a440d509b\") " pod="openshift-dns/node-resolver-gd8hl"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150045    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-run-ovn-kubernetes\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.150105    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150117    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-run-ovn-kubernetes\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150129    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-ovn-node-metrics-cert\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150159    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-ovnkube-script-lib\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.150184    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs podName:ef8b79f7-feb0-46de-898c-1db3ebd142db nodeName:}" failed. No retries permitted until 2026-09-11 16:43:26.650162593 +0000 UTC m=+3.057381236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs") pod "network-metrics-daemon-xnn4g" (UID: "ef8b79f7-feb0-46de-898c-1db3ebd142db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150218    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-var-lib-openvswitch\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/582ece86-42d2-4803-853b-611a440d509b-hosts-file\") pod \"node-resolver-gd8hl\" (UID: \"582ece86-42d2-4803-853b-611a440d509b\") " pod="openshift-dns/node-resolver-gd8hl"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150268    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4604c98e-255c-4fa1-9266-08827d0e3761-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150183    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5e339d2-e61e-4cf6-8f35-aafb644b4136-dbus\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150292    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzf24\" (UniqueName: \"kubernetes.io/projected/4604c98e-255c-4fa1-9266-08827d0e3761-kube-api-access-hzf24\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150317    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-run-netns\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150338    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-log-socket\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150360    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5e339d2-e61e-4cf6-8f35-aafb644b4136-kubelet-config\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-slash\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150405    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-env-overrides\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150429    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f26a8019-80a6-4f41-b39c-8417143279b8-konnectivity-ca\") pod \"konnectivity-agent-jpzqw\" (UID: \"f26a8019-80a6-4f41-b39c-8417143279b8\") " pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150452    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpmzc\" (UniqueName: \"kubernetes.io/projected/ef8b79f7-feb0-46de-898c-1db3ebd142db-kube-api-access-dpmzc\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150485    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-run-systemd\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.150496    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150508    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-cni-bin\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.150547    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret podName:a5e339d2-e61e-4cf6-8f35-aafb644b4136 nodeName:}" failed. No retries permitted until 2026-09-11 16:43:26.650533354 +0000 UTC m=+3.057751986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret") pod "global-pull-secret-syncer-b6fm6" (UID: "a5e339d2-e61e-4cf6-8f35-aafb644b4136") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150567    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-cni-bin\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150569    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4604c98e-255c-4fa1-9266-08827d0e3761-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150577    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6d4q\" (UniqueName: \"kubernetes.io/projected/582ece86-42d2-4803-853b-611a440d509b-kube-api-access-s6d4q\") pod \"node-resolver-gd8hl\" (UID: \"582ece86-42d2-4803-853b-611a440d509b\") " pod="openshift-dns/node-resolver-gd8hl"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150606    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-var-lib-openvswitch\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150635    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-etc-openvswitch\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150656    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/582ece86-42d2-4803-853b-611a440d509b-hosts-file\") pod \"node-resolver-gd8hl\" (UID: \"582ece86-42d2-4803-853b-611a440d509b\") " pod="openshift-dns/node-resolver-gd8hl"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150663    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4604c98e-255c-4fa1-9266-08827d0e3761-cni-binary-copy\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150667    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-run-openvswitch\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150819    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4604c98e-255c-4fa1-9266-08827d0e3761-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150840    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-etc-openvswitch\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150837    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-ovnkube-config\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.151840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150861    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-run-openvswitch\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.152671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4604c98e-255c-4fa1-9266-08827d0e3761-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.152671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150903    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-kubelet\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.152671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.150957    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-slash\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.152671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.151001    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-host-run-netns\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.152671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.151059    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5e339d2-e61e-4cf6-8f35-aafb644b4136-kubelet-config\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:26.152671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.151288    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-ovnkube-script-lib\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.152671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.151299    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-run-systemd\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.152671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.151345    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-log-socket\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.152671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.151406    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-env-overrides\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.152671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.151581    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f26a8019-80a6-4f41-b39c-8417143279b8-konnectivity-ca\") pod \"konnectivity-agent-jpzqw\" (UID: \"f26a8019-80a6-4f41-b39c-8417143279b8\") " pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:26.152671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.152470    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f26a8019-80a6-4f41-b39c-8417143279b8-agent-certs\") pod \"konnectivity-agent-jpzqw\" (UID: \"f26a8019-80a6-4f41-b39c-8417143279b8\") " pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:26.153578 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.153492    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-ovn-node-metrics-cert\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.159130 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.159098    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6d4q\" (UniqueName: \"kubernetes.io/projected/582ece86-42d2-4803-853b-611a440d509b-kube-api-access-s6d4q\") pod \"node-resolver-gd8hl\" (UID: \"582ece86-42d2-4803-853b-611a440d509b\") " pod="openshift-dns/node-resolver-gd8hl"
Sep 11 16:43:26.159795 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.159772    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpmzc\" (UniqueName: \"kubernetes.io/projected/ef8b79f7-feb0-46de-898c-1db3ebd142db-kube-api-access-dpmzc\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:26.159949 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.159928    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-564sd\" (UniqueName: \"kubernetes.io/projected/8ccdd22f-ae97-40ea-a169-7c2c57ac0859-kube-api-access-564sd\") pod \"ovnkube-node-dl5sh\" (UID: \"8ccdd22f-ae97-40ea-a169-7c2c57ac0859\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.160031 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.160010    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:43:26.160031 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.160026    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:43:26.160156 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.160036    2569 projected.go:196] Error preparing data for projected volume kube-api-access-k257s for pod openshift-network-diagnostics/network-check-target-7p27r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:26.160156 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.160108    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s podName:0c446eb1-a854-404f-9106-98d00a6de58c nodeName:}" failed. No retries permitted until 2026-09-11 16:43:26.660095541 +0000 UTC m=+3.067314174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k257s" (UniqueName: "kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s") pod "network-check-target-7p27r" (UID: "0c446eb1-a854-404f-9106-98d00a6de58c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:26.162286 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.162231    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzf24\" (UniqueName: \"kubernetes.io/projected/4604c98e-255c-4fa1-9266-08827d0e3761-kube-api-access-hzf24\") pod \"multus-additional-cni-plugins-6ngsr\" (UID: \"4604c98e-255c-4fa1-9266-08827d0e3761\") " pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.203787 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.203726    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal" event={"ID":"93e5adb587d6c9f5d90710b23c25b76f","Type":"ContainerStarted","Data":"29230d4e55433833501f16a922eff51ca8a4f64d71ff66e4dde8576343043a33"}
Sep 11 16:43:26.205041 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.205012    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal" event={"ID":"f16c66ff742628bc672161cb5b05510a","Type":"ContainerStarted","Data":"f75107a971f8426edca99a7f744b20800dec2b89e9cd7dedcdb7995495b303bf"}
Sep 11 16:43:26.230399 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.230366    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vtmr5"
Sep 11 16:43:26.239118 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.238934    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pnmkk"
Sep 11 16:43:26.252865 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.252836    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw"
Sep 11 16:43:26.257894 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.257871    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8dzr4"
Sep 11 16:43:26.264680 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.264655    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l28sb"
Sep 11 16:43:26.271442 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.271420    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6ngsr"
Sep 11 16:43:26.278195 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.278177    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:26.286857 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.286836    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:26.292566 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.292548    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gd8hl"
Sep 11 16:43:26.654579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.654543    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:26.654759 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.654590    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:26.654759 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.654698    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:26.654759 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.654741    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:26.654936 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.654771    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret podName:a5e339d2-e61e-4cf6-8f35-aafb644b4136 nodeName:}" failed. No retries permitted until 2026-09-11 16:43:27.654752484 +0000 UTC m=+4.061971134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret") pod "global-pull-secret-syncer-b6fm6" (UID: "a5e339d2-e61e-4cf6-8f35-aafb644b4136") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:26.654936 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.654786    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs podName:ef8b79f7-feb0-46de-898c-1db3ebd142db nodeName:}" failed. No retries permitted until 2026-09-11 16:43:27.654780451 +0000 UTC m=+4.061999077 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs") pod "network-metrics-daemon-xnn4g" (UID: "ef8b79f7-feb0-46de-898c-1db3ebd142db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:26.731788 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.731760    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:43:26.755767 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.755730    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k257s\" (UniqueName: \"kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s\") pod \"network-check-target-7p27r\" (UID: \"0c446eb1-a854-404f-9106-98d00a6de58c\") " pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:26.755958 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.755906    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:43:26.755958 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.755930    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:43:26.755958 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.755940    2569 projected.go:196] Error preparing data for projected volume kube-api-access-k257s for pod openshift-network-diagnostics/network-check-target-7p27r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:26.756110 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:26.756006    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s podName:0c446eb1-a854-404f-9106-98d00a6de58c nodeName:}" failed. No retries permitted until 2026-09-11 16:43:27.755989809 +0000 UTC m=+4.163208436 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k257s" (UniqueName: "kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s") pod "network-check-target-7p27r" (UID: "0c446eb1-a854-404f-9106-98d00a6de58c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:26.973401 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.973306    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 16:38:25 +0000 UTC" deadline="2028-02-09 09:52:44.333688626 +0000 UTC"
Sep 11 16:43:26.973401 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:26.973345    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12377h9m17.360348295s"
Sep 11 16:43:27.154224 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:27.154069    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26a8019_80a6_4f41_b39c_8417143279b8.slice/crio-a743abb26058a25d872fe0c6285daaa4aa73e669e62d2bc79cdcc5a63d630f3a WatchSource:0}: Error finding container a743abb26058a25d872fe0c6285daaa4aa73e669e62d2bc79cdcc5a63d630f3a: Status 404 returned error can't find the container with id a743abb26058a25d872fe0c6285daaa4aa73e669e62d2bc79cdcc5a63d630f3a
Sep 11 16:43:27.154972 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:27.154783    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18ea8e25_6023_41a7_8643_9f093ffeab1e.slice/crio-2b4e2897b4a9b12222712fa9cab44cd2de4b0965f96c7466a921e61c4b809793 WatchSource:0}: Error finding container 2b4e2897b4a9b12222712fa9cab44cd2de4b0965f96c7466a921e61c4b809793: Status 404 returned error can't find the container with id 2b4e2897b4a9b12222712fa9cab44cd2de4b0965f96c7466a921e61c4b809793
Sep 11 16:43:27.156017 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:27.155987    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod582ece86_42d2_4803_853b_611a440d509b.slice/crio-6538b9a6fbe996a76fa680c12cf410b804e12c361b1785b0ed6251b083eb3b26 WatchSource:0}: Error finding container 6538b9a6fbe996a76fa680c12cf410b804e12c361b1785b0ed6251b083eb3b26: Status 404 returned error can't find the container with id 6538b9a6fbe996a76fa680c12cf410b804e12c361b1785b0ed6251b083eb3b26
Sep 11 16:43:27.156808 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:27.156785    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4604c98e_255c_4fa1_9266_08827d0e3761.slice/crio-4c8f67cf6aebfcc0477836aa0489c871e07f972e560351c3a55fc58c9c242d01 WatchSource:0}: Error finding container 4c8f67cf6aebfcc0477836aa0489c871e07f972e560351c3a55fc58c9c242d01: Status 404 returned error can't find the container with id 4c8f67cf6aebfcc0477836aa0489c871e07f972e560351c3a55fc58c9c242d01
Sep 11 16:43:27.158861 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:43:27.158286    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ccdd22f_ae97_40ea_a169_7c2c57ac0859.slice/crio-e26de0b982694a6f99b28c3244a7beac8aad3f389549301e88aa1233be91c917 WatchSource:0}: Error finding container e26de0b982694a6f99b28c3244a7beac8aad3f389549301e88aa1233be91c917: Status 404 returned error can't find the container with id e26de0b982694a6f99b28c3244a7beac8aad3f389549301e88aa1233be91c917
Sep 11 16:43:27.198763 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.198582    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:27.198893 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.198627    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:27.198893 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:27.198858    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:27.199005 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:27.198925    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:27.208649 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.208627    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vtmr5" event={"ID":"18ea8e25-6023-41a7-8643-9f093ffeab1e","Type":"ContainerStarted","Data":"2b4e2897b4a9b12222712fa9cab44cd2de4b0965f96c7466a921e61c4b809793"}
Sep 11 16:43:27.209623 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.209604    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gd8hl" event={"ID":"582ece86-42d2-4803-853b-611a440d509b","Type":"ContainerStarted","Data":"6538b9a6fbe996a76fa680c12cf410b804e12c361b1785b0ed6251b083eb3b26"}
Sep 11 16:43:27.210662 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.210637    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jpzqw" event={"ID":"f26a8019-80a6-4f41-b39c-8417143279b8","Type":"ContainerStarted","Data":"a743abb26058a25d872fe0c6285daaa4aa73e669e62d2bc79cdcc5a63d630f3a"}
Sep 11 16:43:27.211772 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.211753    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8dzr4" event={"ID":"b3c0abb4-7329-4793-9028-2090ccf7fa91","Type":"ContainerStarted","Data":"cea7fe747409b8e6a79be4510d0511a468995ab858000c8ddcafd186fa31ba68"}
Sep 11 16:43:27.212829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.212810    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh" event={"ID":"8ccdd22f-ae97-40ea-a169-7c2c57ac0859","Type":"ContainerStarted","Data":"e26de0b982694a6f99b28c3244a7beac8aad3f389549301e88aa1233be91c917"}
Sep 11 16:43:27.214059 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.214040    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ngsr" event={"ID":"4604c98e-255c-4fa1-9266-08827d0e3761","Type":"ContainerStarted","Data":"4c8f67cf6aebfcc0477836aa0489c871e07f972e560351c3a55fc58c9c242d01"}
Sep 11 16:43:27.215194 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.215173    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pnmkk" event={"ID":"d58f1c39-4e28-4ee8-a405-99102811ad67","Type":"ContainerStarted","Data":"7855f44216628c0609805caa965fcae5ad92e2fe396fad1d127c0c5caf658374"}
Sep 11 16:43:27.216425 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.216404    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw" event={"ID":"b6a311db-d6a1-4f9d-8140-b22458cf7b92","Type":"ContainerStarted","Data":"f517abda4a3d628d094aa9d0642b91e8cb9e0df7c8f0d7b15ec463a63c92938e"}
Sep 11 16:43:27.217464 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.217446    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l28sb" event={"ID":"7fa5fa96-b9a1-4392-8189-3944be73baba","Type":"ContainerStarted","Data":"8630a00a60f8885ced149d312990018cc7166a69f013e31fea7bd494f1080368"}
Sep 11 16:43:27.664168 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.663360    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:27.664168 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.663411    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:27.664168 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:27.663584    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:27.664168 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:27.663652    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs podName:ef8b79f7-feb0-46de-898c-1db3ebd142db nodeName:}" failed. No retries permitted until 2026-09-11 16:43:29.663631812 +0000 UTC m=+6.070850457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs") pod "network-metrics-daemon-xnn4g" (UID: "ef8b79f7-feb0-46de-898c-1db3ebd142db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:27.664168 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:27.664059    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:27.664168 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:27.664126    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret podName:a5e339d2-e61e-4cf6-8f35-aafb644b4136 nodeName:}" failed. No retries permitted until 2026-09-11 16:43:29.664109902 +0000 UTC m=+6.071328548 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret") pod "global-pull-secret-syncer-b6fm6" (UID: "a5e339d2-e61e-4cf6-8f35-aafb644b4136") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:27.764824 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.764680    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k257s\" (UniqueName: \"kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s\") pod \"network-check-target-7p27r\" (UID: \"0c446eb1-a854-404f-9106-98d00a6de58c\") " pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:27.765007 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:27.764879    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:43:27.765007 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:27.764904    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:43:27.765007 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:27.764918    2569 projected.go:196] Error preparing data for projected volume kube-api-access-k257s for pod openshift-network-diagnostics/network-check-target-7p27r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:27.765007 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:27.764993    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s podName:0c446eb1-a854-404f-9106-98d00a6de58c nodeName:}" failed. No retries permitted until 2026-09-11 16:43:29.764974465 +0000 UTC m=+6.172193109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k257s" (UniqueName: "kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s") pod "network-check-target-7p27r" (UID: "0c446eb1-a854-404f-9106-98d00a6de58c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:27.973880 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.973767    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 16:38:25 +0000 UTC" deadline="2028-05-20 20:41:25.386088328 +0000 UTC"
Sep 11 16:43:27.973880 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:27.973827    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14811h57m57.412265567s"
Sep 11 16:43:28.200705 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:28.200200    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:28.200705 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:28.200330    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:28.226148 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:28.225693    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal" event={"ID":"93e5adb587d6c9f5d90710b23c25b76f","Type":"ContainerStarted","Data":"aef123cfce4a232d82dd008092e944747ad8b14098c4086b993e10e243386260"}
Sep 11 16:43:28.237694 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:28.237056    2569 generic.go:356] "Generic (PLEG): container finished" podID="f16c66ff742628bc672161cb5b05510a" containerID="85afe7935fd2c5f96084327ea3868e77041bab0ed119b8bfe4ea09d3a02bfc96" exitCode=0
Sep 11 16:43:28.237694 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:28.237127    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal" event={"ID":"f16c66ff742628bc672161cb5b05510a","Type":"ContainerDied","Data":"85afe7935fd2c5f96084327ea3868e77041bab0ed119b8bfe4ea09d3a02bfc96"}
Sep 11 16:43:28.286757 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:28.286446    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal" podStartSLOduration=3.286421654 podStartE2EDuration="3.286421654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:28.250235775 +0000 UTC m=+4.657454447" watchObservedRunningTime="2026-09-11 16:43:28.286421654 +0000 UTC m=+4.693640302"
Sep 11 16:43:29.199615 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:29.199579    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:29.200472 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:29.199716    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:29.200472 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:29.199731    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:29.200472 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:29.199858    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:29.279944 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:29.279167    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal" event={"ID":"f16c66ff742628bc672161cb5b05510a","Type":"ContainerStarted","Data":"bfd9e71a2bd77a71ad2cd5bf0f2895d1722cfadab54318ae3ee62fb45644c930"}
Sep 11 16:43:29.686413 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:29.685547    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:29.686413 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:29.685597    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:29.686413 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:29.685741    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:29.686413 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:29.685806    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs podName:ef8b79f7-feb0-46de-898c-1db3ebd142db nodeName:}" failed. No retries permitted until 2026-09-11 16:43:33.685785632 +0000 UTC m=+10.093004300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs") pod "network-metrics-daemon-xnn4g" (UID: "ef8b79f7-feb0-46de-898c-1db3ebd142db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:29.686413 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:29.686255    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:29.686413 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:29.686312    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret podName:a5e339d2-e61e-4cf6-8f35-aafb644b4136 nodeName:}" failed. No retries permitted until 2026-09-11 16:43:33.686297777 +0000 UTC m=+10.093516409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret") pod "global-pull-secret-syncer-b6fm6" (UID: "a5e339d2-e61e-4cf6-8f35-aafb644b4136") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:29.786653 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:29.786609    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k257s\" (UniqueName: \"kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s\") pod \"network-check-target-7p27r\" (UID: \"0c446eb1-a854-404f-9106-98d00a6de58c\") " pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:29.786859 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:29.786842    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:43:29.786938 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:29.786868    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:43:29.786938 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:29.786882    2569 projected.go:196] Error preparing data for projected volume kube-api-access-k257s for pod openshift-network-diagnostics/network-check-target-7p27r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:29.787041 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:29.786942    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s podName:0c446eb1-a854-404f-9106-98d00a6de58c nodeName:}" failed. No retries permitted until 2026-09-11 16:43:33.786923244 +0000 UTC m=+10.194141872 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k257s" (UniqueName: "kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s") pod "network-check-target-7p27r" (UID: "0c446eb1-a854-404f-9106-98d00a6de58c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:30.200882 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:30.200374    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:30.200882 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:30.200489    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:31.199422 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:31.199386    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:31.199571 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:31.199518    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:31.200034 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:31.199988    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:31.200287 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:31.200243    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:32.198886 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:32.198832    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:32.199379 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:32.199030    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:33.199325 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:33.198799    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:33.199325 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:33.198830    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:33.199325 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:33.198958    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:33.199911 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:33.199401    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:33.718528 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:33.718178    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:33.718528 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:33.718242    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:33.718528 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:33.718370    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:33.718528 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:33.718431    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:33.718528 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:33.718453    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret podName:a5e339d2-e61e-4cf6-8f35-aafb644b4136 nodeName:}" failed. No retries permitted until 2026-09-11 16:43:41.718432887 +0000 UTC m=+18.125651532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret") pod "global-pull-secret-syncer-b6fm6" (UID: "a5e339d2-e61e-4cf6-8f35-aafb644b4136") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:33.718528 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:33.718500    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs podName:ef8b79f7-feb0-46de-898c-1db3ebd142db nodeName:}" failed. No retries permitted until 2026-09-11 16:43:41.718483834 +0000 UTC m=+18.125702465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs") pod "network-metrics-daemon-xnn4g" (UID: "ef8b79f7-feb0-46de-898c-1db3ebd142db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:33.819553 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:33.819514    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k257s\" (UniqueName: \"kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s\") pod \"network-check-target-7p27r\" (UID: \"0c446eb1-a854-404f-9106-98d00a6de58c\") " pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:33.819730 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:33.819709    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:43:33.819818 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:33.819738    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:43:33.819818 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:33.819752    2569 projected.go:196] Error preparing data for projected volume kube-api-access-k257s for pod openshift-network-diagnostics/network-check-target-7p27r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:33.819818 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:33.819808    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s podName:0c446eb1-a854-404f-9106-98d00a6de58c nodeName:}" failed. No retries permitted until 2026-09-11 16:43:41.819792451 +0000 UTC m=+18.227011077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k257s" (UniqueName: "kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s") pod "network-check-target-7p27r" (UID: "0c446eb1-a854-404f-9106-98d00a6de58c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:34.200608 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:34.200549    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:34.201092 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:34.200651    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:35.199565 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:35.198872    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:35.199565 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:35.199000    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:35.199565 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:35.199411    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:35.199565 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:35.199515    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:36.199133 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:36.199097    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:36.199595 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:36.199243    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:37.199693 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:37.199656    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:37.199693 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:37.199679    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:37.200092 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:37.199770    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:37.200092 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:37.199874    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:38.199157 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:38.199118    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:38.199339 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:38.199244    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:39.198968 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:39.198938    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:39.199415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:39.198938    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:39.199415 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:39.199094    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:39.199415 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:39.199132    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:40.199115 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:40.199067    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:40.199541 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:40.199218    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:41.199087 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:41.199041    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:41.199298 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:41.199041    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:41.199298 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:41.199209    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:41.199605 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:41.199310    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:41.773417 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:41.773209    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:41.773599 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:41.773445    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:41.773599 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:41.773379    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:41.773599 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:41.773557    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret podName:a5e339d2-e61e-4cf6-8f35-aafb644b4136 nodeName:}" failed. No retries permitted until 2026-09-11 16:43:57.773536014 +0000 UTC m=+34.180754659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret") pod "global-pull-secret-syncer-b6fm6" (UID: "a5e339d2-e61e-4cf6-8f35-aafb644b4136") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:41.773599 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:41.773575    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:41.773810 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:41.773624    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs podName:ef8b79f7-feb0-46de-898c-1db3ebd142db nodeName:}" failed. No retries permitted until 2026-09-11 16:43:57.773609252 +0000 UTC m=+34.180827877 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs") pod "network-metrics-daemon-xnn4g" (UID: "ef8b79f7-feb0-46de-898c-1db3ebd142db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:41.874510 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:41.874470    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k257s\" (UniqueName: \"kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s\") pod \"network-check-target-7p27r\" (UID: \"0c446eb1-a854-404f-9106-98d00a6de58c\") " pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:41.874695 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:41.874673    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:43:41.874748 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:41.874699    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:43:41.874748 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:41.874713    2569 projected.go:196] Error preparing data for projected volume kube-api-access-k257s for pod openshift-network-diagnostics/network-check-target-7p27r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:41.874839 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:41.874774    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s podName:0c446eb1-a854-404f-9106-98d00a6de58c nodeName:}" failed. No retries permitted until 2026-09-11 16:43:57.87475843 +0000 UTC m=+34.281977080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k257s" (UniqueName: "kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s") pod "network-check-target-7p27r" (UID: "0c446eb1-a854-404f-9106-98d00a6de58c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:42.199376 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:42.199339    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:42.199799 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:42.199482    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:43.199178 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:43.199145    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:43.199407 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:43.199146    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:43.199407 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:43.199260    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:43.199407 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:43.199366    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:44.202404 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:44.202375    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:44.202794 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:44.202478    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:45.199304 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:45.199267    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:45.199437 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:45.199402    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:45.199823 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:45.199798    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:45.199931 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:45.199911    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:45.318882 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:45.318813    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jpzqw" event={"ID":"f26a8019-80a6-4f41-b39c-8417143279b8","Type":"ContainerStarted","Data":"ffdaab01ae5f216a0525adc350d89f5f73304bbf5b9016210d8f52738730cf81"}
Sep 11 16:43:45.320467 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:45.320436    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ngsr" event={"ID":"4604c98e-255c-4fa1-9266-08827d0e3761","Type":"ContainerStarted","Data":"e391bfb3469d3b2bc3f632f4c38465409cf5632b9239eab41e71eac8ab09ff6d"}
Sep 11 16:43:45.334453 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:45.334420    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal" podStartSLOduration=20.334394493 podStartE2EDuration="20.334394493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:29.299260761 +0000 UTC m=+5.706479432" watchObservedRunningTime="2026-09-11 16:43:45.334394493 +0000 UTC m=+21.741613120"
Sep 11 16:43:45.334770 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:45.334748    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jpzqw" podStartSLOduration=10.855241268 podStartE2EDuration="21.334739329s" totalImagesPullingTime="10.479498061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:45.334239278 +0000 UTC m=+21.741457925" watchObservedRunningTime="2026-09-11 16:43:45.334739329 +0000 UTC m=+21.741958006"
Sep 11 16:43:46.199222 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.198957    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:46.199413 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:46.199342    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:46.325416 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.325379    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh" event={"ID":"8ccdd22f-ae97-40ea-a169-7c2c57ac0859","Type":"ContainerStarted","Data":"5b52b4fc0d8e5e5cba2ac10eaadbbe39677347ef0d7ecffe7ad2b63c1102ef43"}
Sep 11 16:43:46.326143 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.325454    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh" event={"ID":"8ccdd22f-ae97-40ea-a169-7c2c57ac0859","Type":"ContainerStarted","Data":"12a2ba9832a37ad99809b35455fc575541e69a4d9197248c3528a594661e434b"}
Sep 11 16:43:46.326143 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.325501    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh" event={"ID":"8ccdd22f-ae97-40ea-a169-7c2c57ac0859","Type":"ContainerStarted","Data":"29f5369158d657f52e8cdd3fb4ab9325a4d02edecfd3ffdbeefe635b1eaf8bcd"}
Sep 11 16:43:46.326143 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.325541    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh" event={"ID":"8ccdd22f-ae97-40ea-a169-7c2c57ac0859","Type":"ContainerStarted","Data":"6da9563fe4c92ed80789bc410232d40e52f5f364dd5d62598cc3e167c507e032"}
Sep 11 16:43:46.326143 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.325586    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh" event={"ID":"8ccdd22f-ae97-40ea-a169-7c2c57ac0859","Type":"ContainerStarted","Data":"de9879fa4a2fb85a828baa75dc1c23710d89eceb428dd3fe2ad8bfee4941a1a5"}
Sep 11 16:43:46.326143 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.325624    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh" event={"ID":"8ccdd22f-ae97-40ea-a169-7c2c57ac0859","Type":"ContainerStarted","Data":"705e3a4949b8369da330a6e21d7588815621297e8bd303381c68582d4e19b033"}
Sep 11 16:43:46.326847 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.326825    2569 generic.go:356] "Generic (PLEG): container finished" podID="4604c98e-255c-4fa1-9266-08827d0e3761" containerID="e391bfb3469d3b2bc3f632f4c38465409cf5632b9239eab41e71eac8ab09ff6d" exitCode=0
Sep 11 16:43:46.326925 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.326898    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ngsr" event={"ID":"4604c98e-255c-4fa1-9266-08827d0e3761","Type":"ContainerDied","Data":"e391bfb3469d3b2bc3f632f4c38465409cf5632b9239eab41e71eac8ab09ff6d"}
Sep 11 16:43:46.328395 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.328376    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw" event={"ID":"b6a311db-d6a1-4f9d-8140-b22458cf7b92","Type":"ContainerStarted","Data":"4efbf8a82b44e70e45c13f5752ea0e7907530c688e55219b31b61e7470bc75a7"}
Sep 11 16:43:46.330016 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.329988    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l28sb" event={"ID":"7fa5fa96-b9a1-4392-8189-3944be73baba","Type":"ContainerStarted","Data":"91181c268917df9c76ce2c0917fd32701f2737bd1e07777a4449cff71a6c771a"}
Sep 11 16:43:46.331486 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.331465    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vtmr5" event={"ID":"18ea8e25-6023-41a7-8643-9f093ffeab1e","Type":"ContainerStarted","Data":"2a85897d5d3f080b0e51cc2f6c16db03a52f3bfef321244f10e567a18e137980"}
Sep 11 16:43:46.332932 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.332896    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gd8hl" event={"ID":"582ece86-42d2-4803-853b-611a440d509b","Type":"ContainerStarted","Data":"9c8c2cb7afba434bd9315b0efb87a0add9b320757f11215bc2aeb15376a9454d"}
Sep 11 16:43:46.334598 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.334574    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8dzr4" event={"ID":"b3c0abb4-7329-4793-9028-2090ccf7fa91","Type":"ContainerStarted","Data":"bbd12c512ae0e5c50fa506d5867bb852ca9b33f3ec550d03cd6adc02a190c722"}
Sep 11 16:43:46.364819 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.364777    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8dzr4" podStartSLOduration=4.422505907 podStartE2EDuration="22.364764866s" totalImagesPullingTime="17.942258959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:46.364438407 +0000 UTC m=+22.771657177" watchObservedRunningTime="2026-09-11 16:43:46.364764866 +0000 UTC m=+22.771983550"
Sep 11 16:43:46.378947 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.378903    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-l28sb" podStartSLOduration=4.436986104 podStartE2EDuration="22.378890914s" totalImagesPullingTime="17.94190481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:46.37872095 +0000 UTC m=+22.785939597" watchObservedRunningTime="2026-09-11 16:43:46.378890914 +0000 UTC m=+22.786109561"
Sep 11 16:43:46.393967 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.393929    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gd8hl" podStartSLOduration=4.441857967 podStartE2EDuration="22.393915828s" totalImagesPullingTime="17.952057861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:46.393374486 +0000 UTC m=+22.800593156" watchObservedRunningTime="2026-09-11 16:43:46.393915828 +0000 UTC m=+22.801134476"
Sep 11 16:43:46.411906 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.411838    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vtmr5" podStartSLOduration=4.155938262 podStartE2EDuration="22.411823285s" totalImagesPullingTime="18.255885023s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:46.411314646 +0000 UTC m=+22.818533313" watchObservedRunningTime="2026-09-11 16:43:46.411823285 +0000 UTC m=+22.819041933"
Sep 11 16:43:46.568067 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.568038    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:46.642639 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.642611    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:46.646968 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.646946    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 16:43:46.745934 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.745858    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:46.996708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:46.996627    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:47.005690 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.005607    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T16:43:46.646964206Z","UUID":"8e1d6d57-d5c2-4b2b-8fe8-410cc8776b5c","Handler":null,"Name":"","Endpoint":""}
Sep 11 16:43:47.007132 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.007112    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 16:43:47.007236 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.007139    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 16:43:47.095409 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.095376    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:47.198990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.198952    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:47.199222 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.199086    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:47.199323 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:47.199296    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:47.199633 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:47.199585    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:47.207471 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.207443    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:47.208069 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.208048    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:47.339047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.338970    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pnmkk" event={"ID":"d58f1c39-4e28-4ee8-a405-99102811ad67","Type":"ContainerStarted","Data":"9ce50eb7da9fea4fd30fc04af7273c455f92b0493de8ae53b2ed001a9cb6c151"}
Sep 11 16:43:47.342018 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.341984    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw" event={"ID":"b6a311db-d6a1-4f9d-8140-b22458cf7b92","Type":"ContainerStarted","Data":"73a32aa1477075f34489e67926ddcb708e4efff76300e6f7919921d9de70d9de"}
Sep 11 16:43:47.342553 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.342511    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:47.343274 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.343254    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jpzqw"
Sep 11 16:43:47.354933 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.354894    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pnmkk" podStartSLOduration=5.413822277 podStartE2EDuration="23.354878898s" totalImagesPullingTime="17.941056621s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:47.354703359 +0000 UTC m=+23.761922026" watchObservedRunningTime="2026-09-11 16:43:47.354878898 +0000 UTC m=+23.762097546"
Sep 11 16:43:47.483725 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.483687    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:47.544563 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:47.544512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:48.199475 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:48.199264    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:48.199675 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:48.199589    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:48.347426 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:48.347384    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh" event={"ID":"8ccdd22f-ae97-40ea-a169-7c2c57ac0859","Type":"ContainerStarted","Data":"b66b24723d617ad500559943d2966f3d62f54bd4f4c617899e1a625efdb6c92d"}
Sep 11 16:43:48.349943 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:48.349913    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw" event={"ID":"b6a311db-d6a1-4f9d-8140-b22458cf7b92","Type":"ContainerStarted","Data":"1c241730808c019a6e9d95fe568260e6956ba004181278fe1e0d3fa1e7823a39"}
Sep 11 16:43:48.354580 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:48.354556    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:48.367898 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:48.367857    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfrvw" podStartSLOduration=4.262883155 podStartE2EDuration="24.367840936s" totalImagesPullingTime="20.104957781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:48.367459944 +0000 UTC m=+24.774678616" watchObservedRunningTime="2026-09-11 16:43:48.367840936 +0000 UTC m=+24.775059584"
Sep 11 16:43:48.455057 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:48.454971    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:49.199672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:49.199638    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:49.199889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:49.199638    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:49.199889 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:49.199772    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:49.199889 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:49.199838    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:49.818714 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:49.818669    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:49.871278 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:49.871244    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:50.199776 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:50.199567    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:50.199776 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:50.199701    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:51.199448 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:51.199421    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:51.199813 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:51.199421    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:51.199813 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:51.199511    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:51.199813 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:51.199596    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:51.360567 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:51.360537    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh" event={"ID":"8ccdd22f-ae97-40ea-a169-7c2c57ac0859","Type":"ContainerStarted","Data":"935d95611f55cef1da3bdf51a956524b272f54f5d7fd713125c99feabe74b671"}
Sep 11 16:43:51.360769 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:51.360749    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:51.362507 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:51.362483    2569 generic.go:356] "Generic (PLEG): container finished" podID="4604c98e-255c-4fa1-9266-08827d0e3761" containerID="7b0a399a1fa71fae957d2f29b824e8212c05c919676c8e5826aeba1f0dbff51f" exitCode=0
Sep 11 16:43:51.362605 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:51.362528    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ngsr" event={"ID":"4604c98e-255c-4fa1-9266-08827d0e3761","Type":"ContainerDied","Data":"7b0a399a1fa71fae957d2f29b824e8212c05c919676c8e5826aeba1f0dbff51f"}
Sep 11 16:43:51.376494 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:51.376471    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:51.388288 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:51.388250    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh" podStartSLOduration=9.426814083 podStartE2EDuration="27.388237284s" totalImagesPullingTime="17.961423201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:51.387643234 +0000 UTC m=+27.794861892" watchObservedRunningTime="2026-09-11 16:43:51.388237284 +0000 UTC m=+27.795455931"
Sep 11 16:43:52.199317 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:52.199281    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:52.199662 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:52.199421    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:52.368304 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:52.368263    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:52.368475 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:52.368450    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:52.383235 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:52.383210    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:43:52.535289 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:52.535207    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xnn4g"]
Sep 11 16:43:52.535435 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:52.535355    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:52.535498 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:52.535474    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:52.538254 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:52.538223    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7p27r"]
Sep 11 16:43:52.538403 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:52.538341    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:52.538478 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:52.538456    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:52.538900 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:52.538872    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b6fm6"]
Sep 11 16:43:52.539100 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:52.538967    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:52.539100 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:52.539054    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:53.371668 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:53.371636    2569 generic.go:356] "Generic (PLEG): container finished" podID="4604c98e-255c-4fa1-9266-08827d0e3761" containerID="5cd911f52661eb0fcd7d15bb9592eeb32da07a1fd5b0a38884cbd4856dc7dbf2" exitCode=0
Sep 11 16:43:53.372262 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:53.371733    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ngsr" event={"ID":"4604c98e-255c-4fa1-9266-08827d0e3761","Type":"ContainerDied","Data":"5cd911f52661eb0fcd7d15bb9592eeb32da07a1fd5b0a38884cbd4856dc7dbf2"}
Sep 11 16:43:54.200402 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:54.200215    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:54.200550 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:54.200276    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:54.200626 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:54.200294    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:54.200626 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:54.200564    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:54.200703 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:54.200646    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:54.200703 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:54.200468    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:55.378250 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:55.378215    2569 generic.go:356] "Generic (PLEG): container finished" podID="4604c98e-255c-4fa1-9266-08827d0e3761" containerID="cc1d9ee5fff152719b204e85354d62184ff3aa6587fcdf29026f8e913167624d" exitCode=0
Sep 11 16:43:55.378250 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:55.378253    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ngsr" event={"ID":"4604c98e-255c-4fa1-9266-08827d0e3761","Type":"ContainerDied","Data":"cc1d9ee5fff152719b204e85354d62184ff3aa6587fcdf29026f8e913167624d"}
Sep 11 16:43:55.638578 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:55.638491    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:55.733379 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:55.733346    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:55.788538 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:55.788481    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:56.198976 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:56.198953    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:56.198976 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:56.198964    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:56.199194 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:56.198956    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:56.199194 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:56.199056    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:56.199194 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:56.199150    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:56.199282 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:56.199236    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:56.812752 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:56.812726    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:56.877799 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:56.877765    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:57.084942 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:57.084862    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:57.181888 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:57.181855    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:57.485906 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:57.485868    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:57.585780 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:57.585736    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:57.737625 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:57.737518    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:57.796599 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:57.796560    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:57.796599 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:57.796590    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:57.796822 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:57.796713    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:57.796822 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:57.796726    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:57.796822 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:57.796783    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs podName:ef8b79f7-feb0-46de-898c-1db3ebd142db nodeName:}" failed. No retries permitted until 2026-09-11 16:44:29.796763809 +0000 UTC m=+66.203982436 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs") pod "network-metrics-daemon-xnn4g" (UID: "ef8b79f7-feb0-46de-898c-1db3ebd142db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:43:57.796822 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:57.796801    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret podName:a5e339d2-e61e-4cf6-8f35-aafb644b4136 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:29.796792356 +0000 UTC m=+66.204010984 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret") pod "global-pull-secret-syncer-b6fm6" (UID: "a5e339d2-e61e-4cf6-8f35-aafb644b4136") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:43:57.837820 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:57.837792    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:57.897737 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:57.897705    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k257s\" (UniqueName: \"kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s\") pod \"network-check-target-7p27r\" (UID: \"0c446eb1-a854-404f-9106-98d00a6de58c\") " pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:57.897916 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:57.897865    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:43:57.897916 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:57.897883    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:43:57.897916 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:57.897895    2569 projected.go:196] Error preparing data for projected volume kube-api-access-k257s for pod openshift-network-diagnostics/network-check-target-7p27r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:57.898051 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:57.897959    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s podName:0c446eb1-a854-404f-9106-98d00a6de58c nodeName:}" failed. No retries permitted until 2026-09-11 16:44:29.897942051 +0000 UTC m=+66.305160685 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k257s" (UniqueName: "kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s") pod "network-check-target-7p27r" (UID: "0c446eb1-a854-404f-9106-98d00a6de58c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:43:58.198757 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:58.198730    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:43:58.198946 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:58.198732    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:43:58.198946 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:58.198846    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:43:58.198946 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:58.198842    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnn4g" podUID="ef8b79f7-feb0-46de-898c-1db3ebd142db"
Sep 11 16:43:58.199137 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:58.198944    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7p27r" podUID="0c446eb1-a854-404f-9106-98d00a6de58c"
Sep 11 16:43:58.199137 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:58.199024    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6fm6" podUID="a5e339d2-e61e-4cf6-8f35-aafb644b4136"
Sep 11 16:43:58.319411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:58.319311    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:58.380154 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:58.380120    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:59.180986 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.180959    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gd8hl_582ece86-42d2-4803-853b-611a440d509b/dns-node-resolver/0.log"
Sep 11 16:43:59.414957 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.414929    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeReady"
Sep 11 16:43:59.415135 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.415060    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 16:43:59.457263 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.457148    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:59.514093 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.514059    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wpfmb"]
Sep 11 16:43:59.515814 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.515778    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.517940 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:59.517914    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"dns-default-metrics-tls\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-dns\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" type="*v1.Secret"
Sep 11 16:43:59.520128 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:59.520097    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"dns-default-wpfmb\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-dns\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="dc500b91-6880-4405-92d9-21e5cc455012" pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.520128 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:59.520101    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"dns-default\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-dns\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-dns\"/\"dns-default\"" type="*v1.ConfigMap"
Sep 11 16:43:59.520128 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:59.520107    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"dns-dockercfg-wbvg9\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-dns\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wbvg9\"" type="*v1.Secret"
Sep 11 16:43:59.529706 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.529680    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mtrfv"]
Sep 11 16:43:59.531778 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.531760    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mtrfv"
Sep 11 16:43:59.538208 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.538184    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 16:43:59.538678 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.538496    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 16:43:59.538678 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.538629    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 16:43:59.541557 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.541535    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7nrk4\""
Sep 11 16:43:59.552712 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.552687    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wpfmb"]
Sep 11 16:43:59.574474 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.574445    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mtrfv"]
Sep 11 16:43:59.592340 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.592312    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d82bq"]
Sep 11 16:43:59.594347 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.594322    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.602739 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.602716    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 16:43:59.602860 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.602750    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 16:43:59.602937 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.602860    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 16:43:59.602937 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.602860    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 16:43:59.603976 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.603962    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9zkcq\""
Sep 11 16:43:59.611379 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.611354    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdqhc\" (UniqueName: \"kubernetes.io/projected/dc500b91-6880-4405-92d9-21e5cc455012-kube-api-access-xdqhc\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.611478 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.611387    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc500b91-6880-4405-92d9-21e5cc455012-tmp-dir\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.611478 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.611426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e42d329c-4c8d-442c-bf85-0a9c6f2d49e3-cert\") pod \"ingress-canary-mtrfv\" (UID: \"e42d329c-4c8d-442c-bf85-0a9c6f2d49e3\") " pod="openshift-ingress-canary/ingress-canary-mtrfv"
Sep 11 16:43:59.611565 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.611494    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dc500b91-6880-4405-92d9-21e5cc455012-config-volume\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.611605 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.611562    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.611605 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.611595    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frcxt\" (UniqueName: \"kubernetes.io/projected/e42d329c-4c8d-442c-bf85-0a9c6f2d49e3-kube-api-access-frcxt\") pod \"ingress-canary-mtrfv\" (UID: \"e42d329c-4c8d-442c-bf85-0a9c6f2d49e3\") " pod="openshift-ingress-canary/ingress-canary-mtrfv"
Sep 11 16:43:59.619822 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.619794    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:59.630027 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.630000    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d82bq"]
Sep 11 16:43:59.712201 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712114    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dc500b91-6880-4405-92d9-21e5cc455012-config-volume\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.712201 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712175    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.712201 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712199    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frcxt\" (UniqueName: \"kubernetes.io/projected/e42d329c-4c8d-442c-bf85-0a9c6f2d49e3-kube-api-access-frcxt\") pod \"ingress-canary-mtrfv\" (UID: \"e42d329c-4c8d-442c-bf85-0a9c6f2d49e3\") " pod="openshift-ingress-canary/ingress-canary-mtrfv"
Sep 11 16:43:59.712479 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712221    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/64f3864a-ce43-4330-b5cc-c43bd0f045e3-crio-socket\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.712479 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712320    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/64f3864a-ce43-4330-b5cc-c43bd0f045e3-data-volume\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.712479 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712359    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q42m4\" (UniqueName: \"kubernetes.io/projected/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-api-access-q42m4\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.712479 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712385    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.712479 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712405    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqhc\" (UniqueName: \"kubernetes.io/projected/dc500b91-6880-4405-92d9-21e5cc455012-kube-api-access-xdqhc\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.712479 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712423    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc500b91-6880-4405-92d9-21e5cc455012-tmp-dir\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.712479 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e42d329c-4c8d-442c-bf85-0a9c6f2d49e3-cert\") pod \"ingress-canary-mtrfv\" (UID: \"e42d329c-4c8d-442c-bf85-0a9c6f2d49e3\") " pod="openshift-ingress-canary/ingress-canary-mtrfv"
Sep 11 16:43:59.712760 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712556    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.712796 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.712773    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc500b91-6880-4405-92d9-21e5cc455012-tmp-dir\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.716456 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.716436    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e42d329c-4c8d-442c-bf85-0a9c6f2d49e3-cert\") pod \"ingress-canary-mtrfv\" (UID: \"e42d329c-4c8d-442c-bf85-0a9c6f2d49e3\") " pod="openshift-ingress-canary/ingress-canary-mtrfv"
Sep 11 16:43:59.732768 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.732742    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frcxt\" (UniqueName: \"kubernetes.io/projected/e42d329c-4c8d-442c-bf85-0a9c6f2d49e3-kube-api-access-frcxt\") pod \"ingress-canary-mtrfv\" (UID: \"e42d329c-4c8d-442c-bf85-0a9c6f2d49e3\") " pod="openshift-ingress-canary/ingress-canary-mtrfv"
Sep 11 16:43:59.735052 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.735031    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdqhc\" (UniqueName: \"kubernetes.io/projected/dc500b91-6880-4405-92d9-21e5cc455012-kube-api-access-xdqhc\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:43:59.813344 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.813310    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/64f3864a-ce43-4330-b5cc-c43bd0f045e3-data-volume\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.813344 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.813346    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q42m4\" (UniqueName: \"kubernetes.io/projected/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-api-access-q42m4\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.813556 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.813370    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.813556 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.813418    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.813556 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.813484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/64f3864a-ce43-4330-b5cc-c43bd0f045e3-crio-socket\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.813556 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:59.813528    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:43:59.813738 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:59.813580    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:43:59.813738 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:59.813616    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm podName:64f3864a-ce43-4330-b5cc-c43bd0f045e3 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:00.31359522 +0000 UTC m=+36.720813862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d82bq" (UID: "64f3864a-ce43-4330-b5cc-c43bd0f045e3") : configmap "kube-rbac-proxy" not found
Sep 11 16:43:59.813738 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:43:59.813639    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls podName:64f3864a-ce43-4330-b5cc-c43bd0f045e3 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:00.313629343 +0000 UTC m=+36.720847974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d82bq" (UID: "64f3864a-ce43-4330-b5cc-c43bd0f045e3") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:43:59.813738 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.813670    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/64f3864a-ce43-4330-b5cc-c43bd0f045e3-crio-socket\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.813738 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.813669    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/64f3864a-ce43-4330-b5cc-c43bd0f045e3-data-volume\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.832989 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.832964    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q42m4\" (UniqueName: \"kubernetes.io/projected/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-api-access-q42m4\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:43:59.841463 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.841439    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mtrfv"
Sep 11 16:43:59.847016 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.846983    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:43:59.903858 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:43:59.903615    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:00.130347 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.130237    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mtrfv"]
Sep 11 16:44:00.183748 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.183719    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l28sb_7fa5fa96-b9a1-4392-8189-3944be73baba/node-ca/0.log"
Sep 11 16:44:00.199179 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.199156    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:44:00.199334 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.199156    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:44:00.199334 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.199294    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:44:00.202795 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.202769    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 16:44:00.203036 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.203018    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 16:44:00.203374 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.203341    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6xdv2\""
Sep 11 16:44:00.206266 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.206247    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 16:44:00.206694 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.206675    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 16:44:00.206774 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.206687    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dw8q7\""
Sep 11 16:44:00.318314 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.318277    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:00.318494 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.318354    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:00.318494 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:00.318415    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:44:00.318494 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:00.318442    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:44:00.318628 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:00.318506    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm podName:64f3864a-ce43-4330-b5cc-c43bd0f045e3 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:01.318483364 +0000 UTC m=+37.725702006 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d82bq" (UID: "64f3864a-ce43-4330-b5cc-c43bd0f045e3") : configmap "kube-rbac-proxy" not found
Sep 11 16:44:00.318628 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:00.318527    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls podName:64f3864a-ce43-4330-b5cc-c43bd0f045e3 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:01.318517883 +0000 UTC m=+37.725736509 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d82bq" (UID: "64f3864a-ce43-4330-b5cc-c43bd0f045e3") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:44:00.448616 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.448482    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:00.712871 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:00.712784    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: failed to sync secret cache: timed out waiting for the condition
Sep 11 16:44:00.712871 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:00.712871    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls podName:dc500b91-6880-4405-92d9-21e5cc455012 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:01.212851541 +0000 UTC m=+37.620070167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls") pod "dns-default-wpfmb" (UID: "dc500b91-6880-4405-92d9-21e5cc455012") : failed to sync secret cache: timed out waiting for the condition
Sep 11 16:44:00.713143 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:00.712790    2569 configmap.go:193] Couldn't get configMap openshift-dns/dns-default: failed to sync configmap cache: timed out waiting for the condition
Sep 11 16:44:00.713143 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:00.712992    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dc500b91-6880-4405-92d9-21e5cc455012-config-volume podName:dc500b91-6880-4405-92d9-21e5cc455012 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:01.212970026 +0000 UTC m=+37.620188661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-volume" (UniqueName: "kubernetes.io/configmap/dc500b91-6880-4405-92d9-21e5cc455012-config-volume") pod "dns-default-wpfmb" (UID: "dc500b91-6880-4405-92d9-21e5cc455012") : failed to sync configmap cache: timed out waiting for the condition
Sep 11 16:44:00.765070 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.765035    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 16:44:00.955168 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:00.955140    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wbvg9\""
Sep 11 16:44:01.107529 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:01.107455    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 16:44:01.225560 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:01.225528    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dc500b91-6880-4405-92d9-21e5cc455012-config-volume\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:44:01.226230 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:01.225590    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:44:01.226230 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:01.225761    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:44:01.226230 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:01.225840    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls podName:dc500b91-6880-4405-92d9-21e5cc455012 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:02.225821477 +0000 UTC m=+38.633040117 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls") pod "dns-default-wpfmb" (UID: "dc500b91-6880-4405-92d9-21e5cc455012") : secret "dns-default-metrics-tls" not found
Sep 11 16:44:01.226230 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:01.226099    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dc500b91-6880-4405-92d9-21e5cc455012-config-volume\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:44:01.326765 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:01.326721    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:01.326962 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:01.326780    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:01.326962 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:01.326842    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:44:01.326962 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:01.326883    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:44:01.326962 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:01.326918    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm podName:64f3864a-ce43-4330-b5cc-c43bd0f045e3 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:03.326895993 +0000 UTC m=+39.734114642 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d82bq" (UID: "64f3864a-ce43-4330-b5cc-c43bd0f045e3") : configmap "kube-rbac-proxy" not found
Sep 11 16:44:01.326962 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:01.326945    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls podName:64f3864a-ce43-4330-b5cc-c43bd0f045e3 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:03.326932744 +0000 UTC m=+39.734151374 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d82bq" (UID: "64f3864a-ce43-4330-b5cc-c43bd0f045e3") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:44:01.438971 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:01.438939    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42d329c_4c8d_442c_bf85_0a9c6f2d49e3.slice/crio-60e6af82f8cf38801698f908dd2fd5d0599abd9c62c79d757b0b159d51cc4b60 WatchSource:0}: Error finding container 60e6af82f8cf38801698f908dd2fd5d0599abd9c62c79d757b0b159d51cc4b60: Status 404 returned error can't find the container with id 60e6af82f8cf38801698f908dd2fd5d0599abd9c62c79d757b0b159d51cc4b60
Sep 11 16:44:02.235839 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:02.235803    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:44:02.236359 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:02.235953    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:44:02.236359 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:02.236016    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls podName:dc500b91-6880-4405-92d9-21e5cc455012 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:04.235998447 +0000 UTC m=+40.643217087 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls") pod "dns-default-wpfmb" (UID: "dc500b91-6880-4405-92d9-21e5cc455012") : secret "dns-default-metrics-tls" not found
Sep 11 16:44:02.398780 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:02.398742    2569 generic.go:356] "Generic (PLEG): container finished" podID="4604c98e-255c-4fa1-9266-08827d0e3761" containerID="a5dfbf775318ee208e5682c99953669dfebd874feb58d9fdc1121d5ab3b632e7" exitCode=0
Sep 11 16:44:02.398972 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:02.398823    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ngsr" event={"ID":"4604c98e-255c-4fa1-9266-08827d0e3761","Type":"ContainerDied","Data":"a5dfbf775318ee208e5682c99953669dfebd874feb58d9fdc1121d5ab3b632e7"}
Sep 11 16:44:02.400351 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:02.400322    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mtrfv" event={"ID":"e42d329c-4c8d-442c-bf85-0a9c6f2d49e3","Type":"ContainerStarted","Data":"60e6af82f8cf38801698f908dd2fd5d0599abd9c62c79d757b0b159d51cc4b60"}
Sep 11 16:44:02.523000 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:02.522971    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:03.345713 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:03.345677    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:03.346328 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:03.345776    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:03.346328 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:03.345859    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:44:03.346328 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:03.345916    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm podName:64f3864a-ce43-4330-b5cc-c43bd0f045e3 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:07.345901353 +0000 UTC m=+43.753119993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d82bq" (UID: "64f3864a-ce43-4330-b5cc-c43bd0f045e3") : configmap "kube-rbac-proxy" not found
Sep 11 16:44:03.346328 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:03.345861    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:44:03.346328 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:03.345990    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls podName:64f3864a-ce43-4330-b5cc-c43bd0f045e3 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:07.345976493 +0000 UTC m=+43.753195134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d82bq" (UID: "64f3864a-ce43-4330-b5cc-c43bd0f045e3") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:44:03.406159 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:03.406126    2569 generic.go:356] "Generic (PLEG): container finished" podID="4604c98e-255c-4fa1-9266-08827d0e3761" containerID="92a447909f82b67b7ded2cdbc53f0bd23861c4f45a8ce577bcaebc58d7dc8b86" exitCode=0
Sep 11 16:44:03.406159 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:03.406169    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ngsr" event={"ID":"4604c98e-255c-4fa1-9266-08827d0e3761","Type":"ContainerDied","Data":"92a447909f82b67b7ded2cdbc53f0bd23861c4f45a8ce577bcaebc58d7dc8b86"}
Sep 11 16:44:03.459535 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:03.459422    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:03.630808 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:03.630771    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:03.732010 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:03.731968    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:03.832175 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:03.832144    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:03.934378 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:03.934301    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:04.095821 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:04.095783    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:04.196439 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:04.196356    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:04.255024 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:04.254988    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:44:04.255195 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:04.255168    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:44:04.255267 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:04.255249    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls podName:dc500b91-6880-4405-92d9-21e5cc455012 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:08.255227725 +0000 UTC m=+44.662446355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls") pod "dns-default-wpfmb" (UID: "dc500b91-6880-4405-92d9-21e5cc455012") : secret "dns-default-metrics-tls" not found
Sep 11 16:44:04.412373 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:04.412340    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ngsr" event={"ID":"4604c98e-255c-4fa1-9266-08827d0e3761","Type":"ContainerStarted","Data":"4c8988602be952ed57e7b432f75347add432719349c2ebb22be40f4615cc47fa"}
Sep 11 16:44:04.433187 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:04.433129    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6ngsr" podStartSLOduration=9.653406492 podStartE2EDuration="40.433107004s" totalImagesPullingTime="30.779700512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:04.432884822 +0000 UTC m=+40.840103472" watchObservedRunningTime="2026-09-11 16:44:04.433107004 +0000 UTC m=+40.840325654"
Sep 11 16:44:04.756001 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:04.755974    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:05.386980 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:05.386950    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:05.416489 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:05.416456    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mtrfv" event={"ID":"e42d329c-4c8d-442c-bf85-0a9c6f2d49e3","Type":"ContainerStarted","Data":"56ee698b3873898dc174ce55d61877e947b30dfca90f461f55b15d4087895dce"}
Sep 11 16:44:05.435647 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:05.435592    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mtrfv" podStartSLOduration=3.170168177 podStartE2EDuration="6.435577577s" totalImagesPullingTime="3.2654094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:05.435019288 +0000 UTC m=+41.842237950" watchObservedRunningTime="2026-09-11 16:44:05.435577577 +0000 UTC m=+41.842796219"
Sep 11 16:44:05.753810 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:05.753732    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:06.155347 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:06.155315    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:06.356779 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:06.356750    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:06.957511 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:06.957477    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:07.227510 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:07.227430    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:07.345510 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:07.345479    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:07.379669 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:07.379639    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:07.379794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:07.379686    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:07.379794 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:07.379758    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:44:07.379794 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:07.379773    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:44:07.379895 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:07.379813    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm podName:64f3864a-ce43-4330-b5cc-c43bd0f045e3 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:15.379799195 +0000 UTC m=+51.787017821 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d82bq" (UID: "64f3864a-ce43-4330-b5cc-c43bd0f045e3") : configmap "kube-rbac-proxy" not found
Sep 11 16:44:07.379895 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:07.379828    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls podName:64f3864a-ce43-4330-b5cc-c43bd0f045e3 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:15.379820175 +0000 UTC m=+51.787038801 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d82bq" (UID: "64f3864a-ce43-4330-b5cc-c43bd0f045e3") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:44:07.447612 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:07.447579    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:07.751647 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:07.751611    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:07.950594 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:07.950561    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:08.007371 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:08.007286    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:08.286892 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:08.286815    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:44:08.289677 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:08.289647    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc500b91-6880-4405-92d9-21e5cc455012-metrics-tls\") pod \"dns-default-wpfmb\" (UID: \"dc500b91-6880-4405-92d9-21e5cc455012\") " pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:44:08.526677 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:08.526641    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:44:08.556385 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:08.556351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:08.760756 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:08.760722    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:08.806353 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:08.806272    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wpfmb"]
Sep 11 16:44:08.809768 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:08.809745    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc500b91_6880_4405_92d9_21e5cc455012.slice/crio-299f84f316de2c5460bd78032b185b784157c1abad2a50b906eed63707d2162c WatchSource:0}: Error finding container 299f84f316de2c5460bd78032b185b784157c1abad2a50b906eed63707d2162c: Status 404 returned error can't find the container with id 299f84f316de2c5460bd78032b185b784157c1abad2a50b906eed63707d2162c
Sep 11 16:44:08.859688 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:08.859649    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:08.920098 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:08.920043    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:09.157974 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:09.157941    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:09.427342 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:09.427264    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wpfmb" event={"ID":"dc500b91-6880-4405-92d9-21e5cc455012","Type":"ContainerStarted","Data":"299f84f316de2c5460bd78032b185b784157c1abad2a50b906eed63707d2162c"}
Sep 11 16:44:09.869154 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:09.869019    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:09.978532 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:09.978496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:10.297514 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:10.297479    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:10.399008 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:10.398968    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:10.562013 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:10.561937    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:10.742625 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:10.742588    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:10.849922 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:10.849850    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:11.154264 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:11.154226    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:11.433420 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:11.433391    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wpfmb" event={"ID":"dc500b91-6880-4405-92d9-21e5cc455012","Type":"ContainerStarted","Data":"7767f319e94a09a25c167f9c6b8180892d50398a531cb5f2372cf7128ecf0642"}
Sep 11 16:44:11.433724 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:11.433471    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wpfmb" event={"ID":"dc500b91-6880-4405-92d9-21e5cc455012","Type":"ContainerStarted","Data":"a18a08a80ff1ed42644ce72d43fbf6935b6e177162bad712cab7fd7293cdfe92"}
Sep 11 16:44:11.789023 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:11.788894    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:12.373135 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:12.373101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:12.436040 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:12.436010    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:44:12.456783 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:12.456729    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wpfmb" podStartSLOduration=11.123091054 podStartE2EDuration="13.456712488s" totalImagesPullingTime="2.333621434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:12.45536823 +0000 UTC m=+48.862586910" watchObservedRunningTime="2026-09-11 16:44:12.456712488 +0000 UTC m=+48.863931168"
Sep 11 16:44:12.553246 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:12.553208    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:12.757022 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:12.756979    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:12.986629 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:12.986599    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:13.766839 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:13.766812    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:13.958535 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:13.958506    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:14.160748 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:14.160719    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:14.259165 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:14.259132    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:14.318723 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:14.318689    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:14.466866 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:14.466791    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:14.695398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:14.695366    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:14.879653 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:14.879616    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:14.978808 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:14.978757    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:15.274697 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:15.274625    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:15.443234 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:15.443204    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:15.443392 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:15.443274    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:15.443766 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:15.443750    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64f3864a-ce43-4330-b5cc-c43bd0f045e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:15.445494 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:15.445476    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64f3864a-ce43-4330-b5cc-c43bd0f045e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d82bq\" (UID: \"64f3864a-ce43-4330-b5cc-c43bd0f045e3\") " pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:15.498217 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:15.498186    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:15.503053 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:15.503031    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d82bq"
Sep 11 16:44:15.556291 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:15.556260    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:15.691038 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:15.691001    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:15.812121 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:15.812030    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d82bq"]
Sep 11 16:44:15.814871 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:15.814844    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f3864a_ce43_4330_b5cc_c43bd0f045e3.slice/crio-bf2d45881897c03ecfaece9f58737e9f107ddd28204e3c7f2f525b417f7159d7 WatchSource:0}: Error finding container bf2d45881897c03ecfaece9f58737e9f107ddd28204e3c7f2f525b417f7159d7: Status 404 returned error can't find the container with id bf2d45881897c03ecfaece9f58737e9f107ddd28204e3c7f2f525b417f7159d7
Sep 11 16:44:15.895780 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:15.895750    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:16.077352 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:16.077261    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:16.446487 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:16.446457    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d82bq" event={"ID":"64f3864a-ce43-4330-b5cc-c43bd0f045e3","Type":"ContainerStarted","Data":"7d8db7ebaa345cac8a9a6763fee8ddde4b0c50178d7b41f92df4132e3f7ecf98"}
Sep 11 16:44:16.446644 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:16.446509    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d82bq" event={"ID":"64f3864a-ce43-4330-b5cc-c43bd0f045e3","Type":"ContainerStarted","Data":"bf2d45881897c03ecfaece9f58737e9f107ddd28204e3c7f2f525b417f7159d7"}
Sep 11 16:44:16.991942 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:16.991914    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:17.089551 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:17.089519    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:17.450543 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:17.450507    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d82bq" event={"ID":"64f3864a-ce43-4330-b5cc-c43bd0f045e3","Type":"ContainerStarted","Data":"9ff5aa8e2ddedfaec0921e43c7ea3f043438815e16b00f615b53c9ee33625322"}
Sep 11 16:44:17.516422 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:17.516374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:17.849674 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:17.849550    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:17.949702 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:17.949670    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:18.050039 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:18.050011    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:18.436255 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:18.436132    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:19.458338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:19.458301    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d82bq" event={"ID":"64f3864a-ce43-4330-b5cc-c43bd0f045e3","Type":"ContainerStarted","Data":"de6f9e0ee048842f3a9bc9c5f312736afdfa9421ecff34ac7ea1a18d7445a4c8"}
Sep 11 16:44:19.497626 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:19.497546    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-d82bq" podStartSLOduration=17.965362053 podStartE2EDuration="20.497529338s" totalImagesPullingTime="2.532167285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:19.496510601 +0000 UTC m=+55.903729280" watchObservedRunningTime="2026-09-11 16:44:19.497529338 +0000 UTC m=+55.904747983"
Sep 11 16:44:21.657948 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.657918    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv"]
Sep 11 16:44:21.659779 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.659764    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv"
Sep 11 16:44:21.662198 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.662177    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-v4c2r\""
Sep 11 16:44:21.662447 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.662430    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 16:44:21.662505 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.662437    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 16:44:21.685942 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.685911    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv"]
Sep 11 16:44:21.686126 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.686105    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/28b211cd-7cd1-4ff3-9280-3001eae82504-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6cmcv\" (UID: \"28b211cd-7cd1-4ff3-9280-3001eae82504\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv"
Sep 11 16:44:21.686205 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.686152    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/28b211cd-7cd1-4ff3-9280-3001eae82504-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-6cmcv\" (UID: \"28b211cd-7cd1-4ff3-9280-3001eae82504\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv"
Sep 11 16:44:21.786777 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.786744    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/28b211cd-7cd1-4ff3-9280-3001eae82504-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6cmcv\" (UID: \"28b211cd-7cd1-4ff3-9280-3001eae82504\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv"
Sep 11 16:44:21.786931 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.786799    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/28b211cd-7cd1-4ff3-9280-3001eae82504-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-6cmcv\" (UID: \"28b211cd-7cd1-4ff3-9280-3001eae82504\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv"
Sep 11 16:44:21.787546 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.787528    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/28b211cd-7cd1-4ff3-9280-3001eae82504-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-6cmcv\" (UID: \"28b211cd-7cd1-4ff3-9280-3001eae82504\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv"
Sep 11 16:44:21.789138 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.789119    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/28b211cd-7cd1-4ff3-9280-3001eae82504-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6cmcv\" (UID: \"28b211cd-7cd1-4ff3-9280-3001eae82504\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv"
Sep 11 16:44:21.849973 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.849939    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-gvcnc"]
Sep 11 16:44:21.851788 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.851772    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-gvcnc"
Sep 11 16:44:21.861267 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.861241    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-lv87z\""
Sep 11 16:44:21.861267 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.861257    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 16:44:21.861446 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.861275    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 16:44:21.862188 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.862170    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps"]
Sep 11 16:44:21.863805 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.863789    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fc5df6dcb-5w5hg"]
Sep 11 16:44:21.863929 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.863911    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps"
Sep 11 16:44:21.865498 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.865483    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.871599 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.871582    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 16:44:21.871738 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.871722    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-g942b\""
Sep 11 16:44:21.874891 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.874876    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-czrjz\""
Sep 11 16:44:21.876352 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.876328    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 16:44:21.876473 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.876458    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 16:44:21.876616 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.876601    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 16:44:21.876714 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.876700    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 16:44:21.877332 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.877318    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 16:44:21.887447 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.887424    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/14501227-b8bc-4233-9fd6-376798609c08-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ngkps\" (UID: \"14501227-b8bc-4233-9fd6-376798609c08\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps"
Sep 11 16:44:21.887537 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.887464    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-config\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.887537 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.887487    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-service-ca\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.887614 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.887532    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24fs8\" (UniqueName: \"kubernetes.io/projected/2ca0adb8-a1e2-42fe-b506-67c40d3de813-kube-api-access-24fs8\") pod \"downloads-76f99584f4-gvcnc\" (UID: \"2ca0adb8-a1e2-42fe-b506-67c40d3de813\") " pod="openshift-console/downloads-76f99584f4-gvcnc"
Sep 11 16:44:21.887614 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.887557    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-serving-cert\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.887711 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.887618    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cssx5\" (UniqueName: \"kubernetes.io/projected/b96801ce-a64a-4b22-831a-fc7ad552c96c-kube-api-access-cssx5\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.887711 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.887684    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-oauth-serving-cert\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.887809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.887756    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-oauth-config\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.900466 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.900443    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-gvcnc"]
Sep 11 16:44:21.906484 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.906455    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc5df6dcb-5w5hg"]
Sep 11 16:44:21.907126 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.907107    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps"]
Sep 11 16:44:21.910383 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.910324    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:21.968364 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.968330    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv"
Sep 11 16:44:21.988756 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.988728    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/14501227-b8bc-4233-9fd6-376798609c08-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ngkps\" (UID: \"14501227-b8bc-4233-9fd6-376798609c08\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps"
Sep 11 16:44:21.988889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.988775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-config\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.988889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.988792    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-service-ca\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.988889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.988809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24fs8\" (UniqueName: \"kubernetes.io/projected/2ca0adb8-a1e2-42fe-b506-67c40d3de813-kube-api-access-24fs8\") pod \"downloads-76f99584f4-gvcnc\" (UID: \"2ca0adb8-a1e2-42fe-b506-67c40d3de813\") " pod="openshift-console/downloads-76f99584f4-gvcnc"
Sep 11 16:44:21.988889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.988825    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-serving-cert\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.988889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.988852    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cssx5\" (UniqueName: \"kubernetes.io/projected/b96801ce-a64a-4b22-831a-fc7ad552c96c-kube-api-access-cssx5\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.988889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.988884    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-oauth-serving-cert\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.989108 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.988922    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-oauth-config\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.989894 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.989863    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-config\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.990028 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.989920    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-service-ca\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.990313 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.990285    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-oauth-serving-cert\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.992035 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.991807    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-oauth-config\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.992148 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.992092    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/14501227-b8bc-4233-9fd6-376798609c08-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ngkps\" (UID: \"14501227-b8bc-4233-9fd6-376798609c08\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps"
Sep 11 16:44:21.992429 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.992411    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"]
Sep 11 16:44:21.992590 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.992571    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-serving-cert\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:21.994796 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.994779    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:21.999343 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.999316    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 16:44:21.999442 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.999417    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ptp2z\""
Sep 11 16:44:21.999520 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:21.999417    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 16:44:22.000326 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.000311    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 16:44:22.012847 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.012827    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 16:44:22.038329 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.038302    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24fs8\" (UniqueName: \"kubernetes.io/projected/2ca0adb8-a1e2-42fe-b506-67c40d3de813-kube-api-access-24fs8\") pod \"downloads-76f99584f4-gvcnc\" (UID: \"2ca0adb8-a1e2-42fe-b506-67c40d3de813\") " pod="openshift-console/downloads-76f99584f4-gvcnc"
Sep 11 16:44:22.038458 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.038393    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cssx5\" (UniqueName: \"kubernetes.io/projected/b96801ce-a64a-4b22-831a-fc7ad552c96c-kube-api-access-cssx5\") pod \"console-6fc5df6dcb-5w5hg\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") " pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:22.041670 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.041642    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"]
Sep 11 16:44:22.077259 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.077197    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:22.089292 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.089268    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e21fe60e-04d0-45fd-8d85-0bb411828a0f-installation-pull-secrets\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.089365 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.089299    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdsmj\" (UniqueName: \"kubernetes.io/projected/e21fe60e-04d0-45fd-8d85-0bb411828a0f-kube-api-access-wdsmj\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.089365 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.089332    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e21fe60e-04d0-45fd-8d85-0bb411828a0f-registry-tls\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.089365 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.089350    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e21fe60e-04d0-45fd-8d85-0bb411828a0f-bound-sa-token\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.089462 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.089365    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e21fe60e-04d0-45fd-8d85-0bb411828a0f-ca-trust-extracted\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.089462 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.089382    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e21fe60e-04d0-45fd-8d85-0bb411828a0f-trusted-ca\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.089462 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.089398    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e21fe60e-04d0-45fd-8d85-0bb411828a0f-image-registry-private-configuration\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.089571 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.089479    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e21fe60e-04d0-45fd-8d85-0bb411828a0f-registry-certificates\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.160985 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.160886    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-gvcnc"
Sep 11 16:44:22.173824 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.173802    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps"
Sep 11 16:44:22.178716 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.178696    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:22.190551 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.190460    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e21fe60e-04d0-45fd-8d85-0bb411828a0f-installation-pull-secrets\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.190551 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.190507    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdsmj\" (UniqueName: \"kubernetes.io/projected/e21fe60e-04d0-45fd-8d85-0bb411828a0f-kube-api-access-wdsmj\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.191837 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.190566    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e21fe60e-04d0-45fd-8d85-0bb411828a0f-registry-tls\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.191837 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.190589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e21fe60e-04d0-45fd-8d85-0bb411828a0f-bound-sa-token\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.191837 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.190613    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e21fe60e-04d0-45fd-8d85-0bb411828a0f-ca-trust-extracted\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.191837 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.190640    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e21fe60e-04d0-45fd-8d85-0bb411828a0f-trusted-ca\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.191837 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.190668    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e21fe60e-04d0-45fd-8d85-0bb411828a0f-image-registry-private-configuration\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.191837 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.190720    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e21fe60e-04d0-45fd-8d85-0bb411828a0f-registry-certificates\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.191837 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.191398    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:22.191837 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.191561    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e21fe60e-04d0-45fd-8d85-0bb411828a0f-registry-certificates\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.191837 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.191670    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e21fe60e-04d0-45fd-8d85-0bb411828a0f-ca-trust-extracted\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.192344 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.192265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e21fe60e-04d0-45fd-8d85-0bb411828a0f-trusted-ca\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.194003 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.193980    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e21fe60e-04d0-45fd-8d85-0bb411828a0f-installation-pull-secrets\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.194270 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.194247    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e21fe60e-04d0-45fd-8d85-0bb411828a0f-image-registry-private-configuration\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.194674 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.194652    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e21fe60e-04d0-45fd-8d85-0bb411828a0f-registry-tls\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.204909 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.204881    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e21fe60e-04d0-45fd-8d85-0bb411828a0f-bound-sa-token\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.212657 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.212559    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdsmj\" (UniqueName: \"kubernetes.io/projected/e21fe60e-04d0-45fd-8d85-0bb411828a0f-kube-api-access-wdsmj\") pod \"image-registry-8f7d7fc8c-4pwmv\" (UID: \"e21fe60e-04d0-45fd-8d85-0bb411828a0f\") " pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.267870 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.267837    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv"]
Sep 11 16:44:22.271086 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:22.271045    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b211cd_7cd1_4ff3_9280_3001eae82504.slice/crio-c7a37454adb5212f48b74e1bb6d582820adb95b9a64d9085c0998e7e473a3b6f WatchSource:0}: Error finding container c7a37454adb5212f48b74e1bb6d582820adb95b9a64d9085c0998e7e473a3b6f: Status 404 returned error can't find the container with id c7a37454adb5212f48b74e1bb6d582820adb95b9a64d9085c0998e7e473a3b6f
Sep 11 16:44:22.291459 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.291434    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:22.311483 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.311459    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:22.382690 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.382659    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:22.438942 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.438853    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-gvcnc"]
Sep 11 16:44:22.440097 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.440061    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wpfmb"
Sep 11 16:44:22.442294 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:22.442273    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ca0adb8_a1e2_42fe_b506_67c40d3de813.slice/crio-b7ec486c7913a3a0437c7c1fec744e02a006738df5c9ec80ce3c2ec3856fc661 WatchSource:0}: Error finding container b7ec486c7913a3a0437c7c1fec744e02a006738df5c9ec80ce3c2ec3856fc661: Status 404 returned error can't find the container with id b7ec486c7913a3a0437c7c1fec744e02a006738df5c9ec80ce3c2ec3856fc661
Sep 11 16:44:22.462584 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.462557    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:22.467412 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.467366    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps"]
Sep 11 16:44:22.471372 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:22.471332    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14501227_b8bc_4233_9fd6_376798609c08.slice/crio-7505fa546ffc0c7ec513d41aebb767625bc6ff92efd05d817825881a310ef95c WatchSource:0}: Error finding container 7505fa546ffc0c7ec513d41aebb767625bc6ff92efd05d817825881a310ef95c: Status 404 returned error can't find the container with id 7505fa546ffc0c7ec513d41aebb767625bc6ff92efd05d817825881a310ef95c
Sep 11 16:44:22.472923 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.472892    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-gvcnc" event={"ID":"2ca0adb8-a1e2-42fe-b506-67c40d3de813","Type":"ContainerStarted","Data":"b7ec486c7913a3a0437c7c1fec744e02a006738df5c9ec80ce3c2ec3856fc661"}
Sep 11 16:44:22.474936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.474911    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc5df6dcb-5w5hg"]
Sep 11 16:44:22.475395 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.475372    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv" event={"ID":"28b211cd-7cd1-4ff3-9280-3001eae82504","Type":"ContainerStarted","Data":"c7a37454adb5212f48b74e1bb6d582820adb95b9a64d9085c0998e7e473a3b6f"}
Sep 11 16:44:22.478619 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:22.478588    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96801ce_a64a_4b22_831a_fc7ad552c96c.slice/crio-f28b17b32c87e0443281bb0562d93e827b31d0e80362618f7a6e106a2b264f7c WatchSource:0}: Error finding container f28b17b32c87e0443281bb0562d93e827b31d0e80362618f7a6e106a2b264f7c: Status 404 returned error can't find the container with id f28b17b32c87e0443281bb0562d93e827b31d0e80362618f7a6e106a2b264f7c
Sep 11 16:44:22.585457 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:22.585424    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"]
Sep 11 16:44:22.589523 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:22.589495    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode21fe60e_04d0_45fd_8d85_0bb411828a0f.slice/crio-3b29cb69d11c0ee95a0fed8b24be9e30dc0a5ebf5d297e3912aa9f5c4847c3f6 WatchSource:0}: Error finding container 3b29cb69d11c0ee95a0fed8b24be9e30dc0a5ebf5d297e3912aa9f5c4847c3f6: Status 404 returned error can't find the container with id 3b29cb69d11c0ee95a0fed8b24be9e30dc0a5ebf5d297e3912aa9f5c4847c3f6
Sep 11 16:44:23.482969 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:23.482880    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc5df6dcb-5w5hg" event={"ID":"b96801ce-a64a-4b22-831a-fc7ad552c96c","Type":"ContainerStarted","Data":"f28b17b32c87e0443281bb0562d93e827b31d0e80362618f7a6e106a2b264f7c"}
Sep 11 16:44:23.487102 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:23.485840    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv" event={"ID":"e21fe60e-04d0-45fd-8d85-0bb411828a0f","Type":"ContainerStarted","Data":"d51d7447b3f2f43e98273ed64fcf40d83765b1adc8ec49196ae0b6c6c7d0d648"}
Sep 11 16:44:23.487102 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:23.485896    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv" event={"ID":"e21fe60e-04d0-45fd-8d85-0bb411828a0f","Type":"ContainerStarted","Data":"3b29cb69d11c0ee95a0fed8b24be9e30dc0a5ebf5d297e3912aa9f5c4847c3f6"}
Sep 11 16:44:23.487102 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:23.486656    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:23.489454 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:23.489391    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps" event={"ID":"14501227-b8bc-4233-9fd6-376798609c08","Type":"ContainerStarted","Data":"7505fa546ffc0c7ec513d41aebb767625bc6ff92efd05d817825881a310ef95c"}
Sep 11 16:44:23.518771 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:23.518708    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv" podStartSLOduration=2.518684258 podStartE2EDuration="2.518684258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:23.517058318 +0000 UTC m=+59.924276991" watchObservedRunningTime="2026-09-11 16:44:23.518684258 +0000 UTC m=+59.925902904"
Sep 11 16:44:24.388408 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.388214    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl5sh"
Sep 11 16:44:24.496960 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.496726    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv" event={"ID":"28b211cd-7cd1-4ff3-9280-3001eae82504","Type":"ContainerStarted","Data":"baee57b7d51e56faa29a50830cff44afbaeeeb126a9937d9dc5401d3c7f25350"}
Sep 11 16:44:24.499366 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.498961    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps" event={"ID":"14501227-b8bc-4233-9fd6-376798609c08","Type":"ContainerStarted","Data":"8182939231b7d3a04501781a4e617202a08ae15b9d5c3a4547954165854af241"}
Sep 11 16:44:24.499366 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.499295    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps"
Sep 11 16:44:24.505112 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.505068    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps"
Sep 11 16:44:24.513288 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.513231    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6cmcv" podStartSLOduration=1.4785423739999999 podStartE2EDuration="3.51321235s" totalImagesPullingTime="2.034669976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:24.513107118 +0000 UTC m=+60.920325797" watchObservedRunningTime="2026-09-11 16:44:24.51321235 +0000 UTC m=+60.920431000"
Sep 11 16:44:24.530816 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.530768    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:24.530992 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.530899    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ngkps" podStartSLOduration=1.693867064 podStartE2EDuration="3.530854176s" totalImagesPullingTime="1.836987112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:24.52965933 +0000 UTC m=+60.936877995" watchObservedRunningTime="2026-09-11 16:44:24.530854176 +0000 UTC m=+60.938073028"
Sep 11 16:44:24.628259 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.628223    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:24.728657 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.728611    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:24.891501 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.891425    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:24.951530 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:24.951498    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:25.043695 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.043642    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:25.088677 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.088649    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"]
Sep 11 16:44:25.091065 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.091041    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.093465 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.093290    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 11 16:44:25.093652 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.093632    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 11 16:44:25.093880 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.093864    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 16:44:25.094091 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.094055    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-c2tj6\""
Sep 11 16:44:25.094411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.094389    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 16:44:25.094598 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.094577    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 16:44:25.102261 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.102200    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"]
Sep 11 16:44:25.118676 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.118647    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3571c72c-acb0-4605-836a-3f8711ee488e-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.118810 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.118730    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl24q\" (UniqueName: \"kubernetes.io/projected/3571c72c-acb0-4605-836a-3f8711ee488e-kube-api-access-tl24q\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.118810 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.118779    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3571c72c-acb0-4605-836a-3f8711ee488e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.118941 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.118826    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3571c72c-acb0-4605-836a-3f8711ee488e-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.120053 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.120025    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:25.219769 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.219682    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl24q\" (UniqueName: \"kubernetes.io/projected/3571c72c-acb0-4605-836a-3f8711ee488e-kube-api-access-tl24q\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.219769 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.219754    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3571c72c-acb0-4605-836a-3f8711ee488e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.219976 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.219809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3571c72c-acb0-4605-836a-3f8711ee488e-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.219976 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.219873    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3571c72c-acb0-4605-836a-3f8711ee488e-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.220126 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:25.220005    2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 11 16:44:25.220186 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:25.220179    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3571c72c-acb0-4605-836a-3f8711ee488e-prometheus-operator-tls podName:3571c72c-acb0-4605-836a-3f8711ee488e nodeName:}" failed. No retries permitted until 2026-09-11 16:44:25.720050508 +0000 UTC m=+62.127269137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3571c72c-acb0-4605-836a-3f8711ee488e-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-x8lmv" (UID: "3571c72c-acb0-4605-836a-3f8711ee488e") : secret "prometheus-operator-tls" not found
Sep 11 16:44:25.220937 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.220887    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3571c72c-acb0-4605-836a-3f8711ee488e-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.223136 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.223069    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3571c72c-acb0-4605-836a-3f8711ee488e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.232365 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.232339    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl24q\" (UniqueName: \"kubernetes.io/projected/3571c72c-acb0-4605-836a-3f8711ee488e-kube-api-access-tl24q\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.549164 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.549056    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:25.724960 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.724834    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3571c72c-acb0-4605-836a-3f8711ee488e-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:25.727814 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:25.727772    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3571c72c-acb0-4605-836a-3f8711ee488e-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-x8lmv\" (UID: \"3571c72c-acb0-4605-836a-3f8711ee488e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:26.004191 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:26.004153    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"
Sep 11 16:44:26.099002 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:26.098975    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:26.203444 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:26.203410    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:26.284531 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:26.284458    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv"]
Sep 11 16:44:26.287380 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:26.287353    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3571c72c_acb0_4605_836a_3f8711ee488e.slice/crio-d4fb43c41c60335c561a3be27dc9f40124857ff43e5ba0d2e3f6d8502a7f0531 WatchSource:0}: Error finding container d4fb43c41c60335c561a3be27dc9f40124857ff43e5ba0d2e3f6d8502a7f0531: Status 404 returned error can't find the container with id d4fb43c41c60335c561a3be27dc9f40124857ff43e5ba0d2e3f6d8502a7f0531
Sep 11 16:44:26.508488 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:26.508452    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc5df6dcb-5w5hg" event={"ID":"b96801ce-a64a-4b22-831a-fc7ad552c96c","Type":"ContainerStarted","Data":"2b6442564c7713e7cc380b6a085bbf3bd637961da4efc16718bf0963af78434e"}
Sep 11 16:44:26.509953 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:26.509923    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv" event={"ID":"3571c72c-acb0-4605-836a-3f8711ee488e","Type":"ContainerStarted","Data":"d4fb43c41c60335c561a3be27dc9f40124857ff43e5ba0d2e3f6d8502a7f0531"}
Sep 11 16:44:26.527769 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:26.527711    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fc5df6dcb-5w5hg" podStartSLOduration=1.968055583 podStartE2EDuration="5.527695609s" totalImagesPullingTime="3.559640026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:26.526654209 +0000 UTC m=+62.933872882" watchObservedRunningTime="2026-09-11 16:44:26.527695609 +0000 UTC m=+62.934914301"
Sep 11 16:44:28.130786 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:28.130739    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:28.520035 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:28.519934    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv" event={"ID":"3571c72c-acb0-4605-836a-3f8711ee488e","Type":"ContainerStarted","Data":"2f9f6d4a7320db70dcaa8788b99d2895d81fbc086451d0e71201352dee783aed"}
Sep 11 16:44:28.520035 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:28.519997    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv" event={"ID":"3571c72c-acb0-4605-836a-3f8711ee488e","Type":"ContainerStarted","Data":"4d3e22849ab93bdc9dd515437bb010486f67905c4b33a9a27115fe7529b5ee6e"}
Sep 11 16:44:29.506998 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.506959    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:29.581213 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.581168    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:29.863556 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.863470    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:44:29.863718 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.863639    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:44:29.866709 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.866681    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 16:44:29.866836 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.866723    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 16:44:29.877109 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.877068    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5e339d2-e61e-4cf6-8f35-aafb644b4136-original-pull-secret\") pod \"global-pull-secret-syncer-b6fm6\" (UID: \"a5e339d2-e61e-4cf6-8f35-aafb644b4136\") " pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:44:29.877213 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.877113    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8b79f7-feb0-46de-898c-1db3ebd142db-metrics-certs\") pod \"network-metrics-daemon-xnn4g\" (UID: \"ef8b79f7-feb0-46de-898c-1db3ebd142db\") " pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:44:29.910902 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.910873    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6fm6"
Sep 11 16:44:29.926424 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.926400    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dw8q7\""
Sep 11 16:44:29.934595 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.934523    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnn4g"
Sep 11 16:44:29.965107 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.965017    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k257s\" (UniqueName: \"kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s\") pod \"network-check-target-7p27r\" (UID: \"0c446eb1-a854-404f-9106-98d00a6de58c\") " pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:44:29.968118 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.967870    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 16:44:29.978372 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.977975    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 16:44:29.990686 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:29.990537    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k257s\" (UniqueName: \"kubernetes.io/projected/0c446eb1-a854-404f-9106-98d00a6de58c-kube-api-access-k257s\") pod \"network-check-target-7p27r\" (UID: \"0c446eb1-a854-404f-9106-98d00a6de58c\") " pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:44:30.195909 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.195848    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-x8lmv" podStartSLOduration=3.668038366 podStartE2EDuration="5.195828081s" totalImagesPullingTime="1.527789715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:28.538201751 +0000 UTC m=+64.945420425" watchObservedRunningTime="2026-09-11 16:44:30.195828081 +0000 UTC m=+66.603046729"
Sep 11 16:44:30.196149 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.196109    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:30.196388 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.196307    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b6fm6"]
Sep 11 16:44:30.199194 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:30.199165    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e339d2_e61e_4cf6_8f35_aafb644b4136.slice/crio-b64c16d6500b07094ac9e0848994616b9586abe51e70bc12137e066544f06548 WatchSource:0}: Error finding container b64c16d6500b07094ac9e0848994616b9586abe51e70bc12137e066544f06548: Status 404 returned error can't find the container with id b64c16d6500b07094ac9e0848994616b9586abe51e70bc12137e066544f06548
Sep 11 16:44:30.221164 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.221135    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6xdv2\""
Sep 11 16:44:30.228191 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.228156    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xnn4g"]
Sep 11 16:44:30.228673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.228652    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:44:30.231544 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:30.231514    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8b79f7_feb0_46de_898c_1db3ebd142db.slice/crio-34ea22b1895a9c4686a7866794f7f2a9f7e75117dcb33566508139aa20208c58 WatchSource:0}: Error finding container 34ea22b1895a9c4686a7866794f7f2a9f7e75117dcb33566508139aa20208c58: Status 404 returned error can't find the container with id 34ea22b1895a9c4686a7866794f7f2a9f7e75117dcb33566508139aa20208c58
Sep 11 16:44:30.405150 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.405119    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:30.434751 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.434725    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"]
Sep 11 16:44:30.439151 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.439130    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.441317 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.441294    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-b4sdb\""
Sep 11 16:44:30.441442 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.441294    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 16:44:30.441442 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.441293    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 16:44:30.447447 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.447142    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"]
Sep 11 16:44:30.473576 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.473549    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8thxd"]
Sep 11 16:44:30.476732 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.476711    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.478754 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.478729    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 16:44:30.478889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.478791    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 16:44:30.479090 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.479050    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 16:44:30.479190 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.479175    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7dzm5\""
Sep 11 16:44:30.492147 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.492124    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:30.512716 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.512545    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7p27r"]
Sep 11 16:44:30.515438 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:30.515412    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c446eb1_a854_404f_9106_98d00a6de58c.slice/crio-523c5334fd7b14e6f15c0c7a726d59b5a7f56c5eb3d46274ffefcb4621af2ea4 WatchSource:0}: Error finding container 523c5334fd7b14e6f15c0c7a726d59b5a7f56c5eb3d46274ffefcb4621af2ea4: Status 404 returned error can't find the container with id 523c5334fd7b14e6f15c0c7a726d59b5a7f56c5eb3d46274ffefcb4621af2ea4
Sep 11 16:44:30.529243 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.529211    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7p27r" event={"ID":"0c446eb1-a854-404f-9106-98d00a6de58c","Type":"ContainerStarted","Data":"523c5334fd7b14e6f15c0c7a726d59b5a7f56c5eb3d46274ffefcb4621af2ea4"}
Sep 11 16:44:30.530941 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.530909    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xnn4g" event={"ID":"ef8b79f7-feb0-46de-898c-1db3ebd142db","Type":"ContainerStarted","Data":"34ea22b1895a9c4686a7866794f7f2a9f7e75117dcb33566508139aa20208c58"}
Sep 11 16:44:30.532407 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.532379    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b6fm6" event={"ID":"a5e339d2-e61e-4cf6-8f35-aafb644b4136","Type":"ContainerStarted","Data":"b64c16d6500b07094ac9e0848994616b9586abe51e70bc12137e066544f06548"}
Sep 11 16:44:30.553839 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.553807    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:30.569522 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569493    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e1dfad2-13dd-4480-b68b-4a711edf9c29-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.569669 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569538    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.569669 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569565    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-sys\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.569669 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569638    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e1dfad2-13dd-4480-b68b-4a711edf9c29-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.569800 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1dfad2-13dd-4480-b68b-4a711edf9c29-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.569800 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569709    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmmpd\" (UniqueName: \"kubernetes.io/projected/9e1dfad2-13dd-4480-b68b-4a711edf9c29-kube-api-access-zmmpd\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.569800 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569763    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-metrics-client-ca\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.569800 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569785    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-root\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.569994 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569801    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-tls\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.569994 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569820    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvhsn\" (UniqueName: \"kubernetes.io/projected/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-kube-api-access-vvhsn\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.569994 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569878    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-textfile\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.569994 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569925    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-wtmp\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.569994 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.569964    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-accelerators-collector-config\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.655415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.655380    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:30.671224 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671192    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-root\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.671224 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-tls\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.671469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvhsn\" (UniqueName: \"kubernetes.io/projected/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-kube-api-access-vvhsn\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.671469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671286    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-textfile\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.671469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671333    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-root\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.671469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671314    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-wtmp\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.671469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671399    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-accelerators-collector-config\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.671469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671465    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e1dfad2-13dd-4480-b68b-4a711edf9c29-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.671763 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671516    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.671763 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671549    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-sys\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.671763 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671585    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e1dfad2-13dd-4480-b68b-4a711edf9c29-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.671763 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671614    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1dfad2-13dd-4480-b68b-4a711edf9c29-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.671763 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671639    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmmpd\" (UniqueName: \"kubernetes.io/projected/9e1dfad2-13dd-4480-b68b-4a711edf9c29-kube-api-access-zmmpd\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.671763 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.671667    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-metrics-client-ca\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.673131 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.672257    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-wtmp\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.673131 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.672278    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-metrics-client-ca\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.673131 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.672345    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-sys\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.673131 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.672525    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-textfile\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.673131 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:30.672627    2569 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 16:44:30.673131 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:44:30.672682    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1dfad2-13dd-4480-b68b-4a711edf9c29-openshift-state-metrics-tls podName:9e1dfad2-13dd-4480-b68b-4a711edf9c29 nodeName:}" failed. No retries permitted until 2026-09-11 16:44:31.172664275 +0000 UTC m=+67.579882917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9e1dfad2-13dd-4480-b68b-4a711edf9c29-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-2wnhh" (UID: "9e1dfad2-13dd-4480-b68b-4a711edf9c29") : secret "openshift-state-metrics-tls" not found
Sep 11 16:44:30.673131 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.672908    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-accelerators-collector-config\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.673131 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.672997    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e1dfad2-13dd-4480-b68b-4a711edf9c29-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.675433 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.674426    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-tls\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.675433 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.674733    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.675433 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.675365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e1dfad2-13dd-4480-b68b-4a711edf9c29-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.681501 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.681476    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvhsn\" (UniqueName: \"kubernetes.io/projected/1189ee6b-cf59-424e-82bc-e1b7d8345eaf-kube-api-access-vvhsn\") pod \"node-exporter-8thxd\" (UID: \"1189ee6b-cf59-424e-82bc-e1b7d8345eaf\") " pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.683960 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.683920    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmmpd\" (UniqueName: \"kubernetes.io/projected/9e1dfad2-13dd-4480-b68b-4a711edf9c29-kube-api-access-zmmpd\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:30.757679 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.757569    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:30.788103 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.788057    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8thxd"
Sep 11 16:44:30.802267 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:30.802236    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1189ee6b_cf59_424e_82bc_e1b7d8345eaf.slice/crio-9a68b2e080e8e8eefb38bfc322028c4870d1bfa1475b2de4185ec8962ca2bb27 WatchSource:0}: Error finding container 9a68b2e080e8e8eefb38bfc322028c4870d1bfa1475b2de4185ec8962ca2bb27: Status 404 returned error can't find the container with id 9a68b2e080e8e8eefb38bfc322028c4870d1bfa1475b2de4185ec8962ca2bb27
Sep 11 16:44:30.855197 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:30.855161    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:31.017266 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:31.017187    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:31.176004 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:31.175966    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1dfad2-13dd-4480-b68b-4a711edf9c29-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:31.191156 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:31.191122    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1dfad2-13dd-4480-b68b-4a711edf9c29-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-2wnhh\" (UID: \"9e1dfad2-13dd-4480-b68b-4a711edf9c29\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:31.206543 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:31.206511    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:31.350470 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:31.350386    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"
Sep 11 16:44:31.424509 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:31.424414    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:31.522214 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:31.522125    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:31.540820 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:31.540781    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8thxd" event={"ID":"1189ee6b-cf59-424e-82bc-e1b7d8345eaf","Type":"ContainerStarted","Data":"9a68b2e080e8e8eefb38bfc322028c4870d1bfa1475b2de4185ec8962ca2bb27"}
Sep 11 16:44:31.643632 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:31.643545    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:31.715129 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:31.715096    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh"]
Sep 11 16:44:31.719878 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:31.719842    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:31.721560 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:31.721532    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1dfad2_13dd_4480_b68b_4a711edf9c29.slice/crio-65b1c1f891dc107ff12a163f6d1a6efa46079274af49668406039f2e5952af6d WatchSource:0}: Error finding container 65b1c1f891dc107ff12a163f6d1a6efa46079274af49668406039f2e5952af6d: Status 404 returned error can't find the container with id 65b1c1f891dc107ff12a163f6d1a6efa46079274af49668406039f2e5952af6d
Sep 11 16:44:32.180552 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.180519    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:32.182770 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.182700    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:32.188825 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.188611    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:32.461929 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.461902    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7ff974997c-trpkx"]
Sep 11 16:44:32.467336 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.467307    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.469618 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.469594    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 16:44:32.470046 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.470024    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8ot89d2uiaajp\""
Sep 11 16:44:32.471274 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.470148    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6vj4r\""
Sep 11 16:44:32.471274 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.470242    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 16:44:32.471274 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.470256    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 16:44:32.471274 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.470360    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 16:44:32.471274 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.470469    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 16:44:32.477690 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.477467    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7ff974997c-trpkx"]
Sep 11 16:44:32.550461 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.550421    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xnn4g" event={"ID":"ef8b79f7-feb0-46de-898c-1db3ebd142db","Type":"ContainerStarted","Data":"9561e0dd41cdc788a5adf23a1d7ed0aa07de7c40f6042caa18987d2b5ea6e703"}
Sep 11 16:44:32.550985 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.550489    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xnn4g" event={"ID":"ef8b79f7-feb0-46de-898c-1db3ebd142db","Type":"ContainerStarted","Data":"07c44d6e36084ec7e0731f1925d848a84575c0db35d5e248180ca662b1914491"}
Sep 11 16:44:32.553884 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.553830    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8thxd" event={"ID":"1189ee6b-cf59-424e-82bc-e1b7d8345eaf","Type":"ContainerStarted","Data":"9670ed3e7eed92d621554bdad1d6b4008d519ee8c6ccc5e98b9473cb8cc9b6ad"}
Sep 11 16:44:32.557625 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.557499    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh" event={"ID":"9e1dfad2-13dd-4480-b68b-4a711edf9c29","Type":"ContainerStarted","Data":"ee73e449df0a22459349dcd27bed8cfd28509f4de56523a9f77017af7d844539"}
Sep 11 16:44:32.557625 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.557554    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh" event={"ID":"9e1dfad2-13dd-4480-b68b-4a711edf9c29","Type":"ContainerStarted","Data":"6f7dac2c744af824cc56184e8106529e668b94eae11533643f39ea6ab0ddcbcf"}
Sep 11 16:44:32.557625 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.557583    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh" event={"ID":"9e1dfad2-13dd-4480-b68b-4a711edf9c29","Type":"ContainerStarted","Data":"65b1c1f891dc107ff12a163f6d1a6efa46079274af49668406039f2e5952af6d"}
Sep 11 16:44:32.563378 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.563334    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:44:32.569562 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.568804    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xnn4g" podStartSLOduration=67.323382379 podStartE2EDuration="1m8.568786185s" totalImagesPullingTime="1.245403806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:32.568291101 +0000 UTC m=+68.975509761" watchObservedRunningTime="2026-09-11 16:44:32.568786185 +0000 UTC m=+68.976004835"
Sep 11 16:44:32.591495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.591150    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.591495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.591195    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd73d7c4-ea76-4820-8c34-c74f014fa722-metrics-client-ca\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.591495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.591236    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-tls\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.591495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.591263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-grpc-tls\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.591495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.591292    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.591495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.591335    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.591495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.591372    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzvm2\" (UniqueName: \"kubernetes.io/projected/fd73d7c4-ea76-4820-8c34-c74f014fa722-kube-api-access-kzvm2\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.591495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.591398    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.684095 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.683987    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:32.694351 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.692793    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.694351 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.692867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzvm2\" (UniqueName: \"kubernetes.io/projected/fd73d7c4-ea76-4820-8c34-c74f014fa722-kube-api-access-kzvm2\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.694351 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.692897    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.694351 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.693023    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.694351 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.693054    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd73d7c4-ea76-4820-8c34-c74f014fa722-metrics-client-ca\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.694351 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.693192    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-tls\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.694351 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.693289    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-grpc-tls\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.694351 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.693322    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.695657 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.695126    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd73d7c4-ea76-4820-8c34-c74f014fa722-metrics-client-ca\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.703111 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.702578    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-grpc-tls\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.703111 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.702998    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.703836 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.703328    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.703836 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.703509    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-tls\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.703836 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.703763    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.704290 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.704267    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd73d7c4-ea76-4820-8c34-c74f014fa722-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.705135 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.705037    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzvm2\" (UniqueName: \"kubernetes.io/projected/fd73d7c4-ea76-4820-8c34-c74f014fa722-kube-api-access-kzvm2\") pod \"thanos-querier-7ff974997c-trpkx\" (UID: \"fd73d7c4-ea76-4820-8c34-c74f014fa722\") " pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:32.786279 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.786243    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:32.788687 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:32.788648    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:33.101162 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.101022    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7ff974997c-trpkx"]
Sep 11 16:44:33.110170 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.110049    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:33.235258 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:33.235216    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd73d7c4_ea76_4820_8c34_c74f014fa722.slice/crio-46d5e0ce72f45a9a80b02a08b0051762a026083774a94d2a3e1a0b7681a13c4d WatchSource:0}: Error finding container 46d5e0ce72f45a9a80b02a08b0051762a026083774a94d2a3e1a0b7681a13c4d: Status 404 returned error can't find the container with id 46d5e0ce72f45a9a80b02a08b0051762a026083774a94d2a3e1a0b7681a13c4d
Sep 11 16:44:33.401468 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.401388    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:33.464148 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.464047    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:33.562371 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.562272    2569 generic.go:356] "Generic (PLEG): container finished" podID="1189ee6b-cf59-424e-82bc-e1b7d8345eaf" containerID="9670ed3e7eed92d621554bdad1d6b4008d519ee8c6ccc5e98b9473cb8cc9b6ad" exitCode=0
Sep 11 16:44:33.562371 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.562367    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8thxd" event={"ID":"1189ee6b-cf59-424e-82bc-e1b7d8345eaf","Type":"ContainerDied","Data":"9670ed3e7eed92d621554bdad1d6b4008d519ee8c6ccc5e98b9473cb8cc9b6ad"}
Sep 11 16:44:33.565874 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.565819    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh" event={"ID":"9e1dfad2-13dd-4480-b68b-4a711edf9c29","Type":"ContainerStarted","Data":"f953a1acf84216f764a769c0e7012a88de60c334668f6a9ee9c34ca1849b454b"}
Sep 11 16:44:33.567709 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.567682    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx" event={"ID":"fd73d7c4-ea76-4820-8c34-c74f014fa722","Type":"ContainerStarted","Data":"46d5e0ce72f45a9a80b02a08b0051762a026083774a94d2a3e1a0b7681a13c4d"}
Sep 11 16:44:33.585586 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.584089    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:33.614902 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.612597    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2wnhh" podStartSLOduration=2.230594328 podStartE2EDuration="3.612576263s" totalImagesPullingTime="1.381981935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:33.612323563 +0000 UTC m=+70.019542290" watchObservedRunningTime="2026-09-11 16:44:33.612576263 +0000 UTC m=+70.019794912"
Sep 11 16:44:33.715851 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.715815    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:33.870464 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.870370    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:33.929335 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:33.929292    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:34.897047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:34.897007    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:34.952572 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:34.952507    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:35.213130 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.213032    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:35.220107 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.219980    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"]
Sep 11 16:44:35.252558 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.252517    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"]
Sep 11 16:44:35.252734 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.252618    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"
Sep 11 16:44:35.254928 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.254905    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 11 16:44:35.255349 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.255028    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2qfsg\""
Sep 11 16:44:35.276589 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.276554    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:35.316716 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.316679    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc0b8a1a-a005-4d8d-a407-a93598edfc6d-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-wxsp7\" (UID: \"bc0b8a1a-a005-4d8d-a407-a93598edfc6d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"
Sep 11 16:44:35.316891 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.316826    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bh2l\" (UniqueName: \"kubernetes.io/projected/bc0b8a1a-a005-4d8d-a407-a93598edfc6d-kube-api-access-8bh2l\") pod \"monitoring-plugin-5d85778b9d-wxsp7\" (UID: \"bc0b8a1a-a005-4d8d-a407-a93598edfc6d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"
Sep 11 16:44:35.340901 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.340867    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:35.417459 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.417411    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bh2l\" (UniqueName: \"kubernetes.io/projected/bc0b8a1a-a005-4d8d-a407-a93598edfc6d-kube-api-access-8bh2l\") pod \"monitoring-plugin-5d85778b9d-wxsp7\" (UID: \"bc0b8a1a-a005-4d8d-a407-a93598edfc6d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"
Sep 11 16:44:35.417639 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.417491    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc0b8a1a-a005-4d8d-a407-a93598edfc6d-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-wxsp7\" (UID: \"bc0b8a1a-a005-4d8d-a407-a93598edfc6d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"
Sep 11 16:44:35.420673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.420637    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc0b8a1a-a005-4d8d-a407-a93598edfc6d-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-wxsp7\" (UID: \"bc0b8a1a-a005-4d8d-a407-a93598edfc6d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"
Sep 11 16:44:35.428328 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.428302    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bh2l\" (UniqueName: \"kubernetes.io/projected/bc0b8a1a-a005-4d8d-a407-a93598edfc6d-kube-api-access-8bh2l\") pod \"monitoring-plugin-5d85778b9d-wxsp7\" (UID: \"bc0b8a1a-a005-4d8d-a407-a93598edfc6d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"
Sep 11 16:44:35.564683 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:35.564604    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"
Sep 11 16:44:36.331471 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.331442    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85d5cc66cb-92qfs"]
Sep 11 16:44:36.348820 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.348781    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d5cc66cb-92qfs"]
Sep 11 16:44:36.348992 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.348920    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.356574 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.356549    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 16:44:36.427452 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.427406    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-serving-cert\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.427452 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.427453    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-trusted-ca-bundle\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.427652 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.427538    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-oauth-serving-cert\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.427652 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.427582    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wntgb\" (UniqueName: \"kubernetes.io/projected/ce28c939-7365-44a2-a4e4-634e634acb29-kube-api-access-wntgb\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.427652 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.427608    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-console-config\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.427652 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.427638    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-service-ca\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.427897 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.427797    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-oauth-config\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.528945 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.528905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-oauth-config\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.529148 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.528978    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-serving-cert\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.529148 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.529004    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-trusted-ca-bundle\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.529148 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.529071    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-oauth-serving-cert\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.529148 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.529113    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wntgb\" (UniqueName: \"kubernetes.io/projected/ce28c939-7365-44a2-a4e4-634e634acb29-kube-api-access-wntgb\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.529148 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.529141    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-console-config\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.529335 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.529175    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-service-ca\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.529881 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.529853    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-oauth-serving-cert\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.530009 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.529884    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-service-ca\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.530009 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.529968    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-trusted-ca-bundle\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.530185 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.530160    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-console-config\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.532995 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.532957    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-oauth-config\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.533415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.533396    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-serving-cert\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.538691 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.538666    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wntgb\" (UniqueName: \"kubernetes.io/projected/ce28c939-7365-44a2-a4e4-634e634acb29-kube-api-access-wntgb\") pod \"console-85d5cc66cb-92qfs\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") " pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.660379 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.660337    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:36.673825 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.673793    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:36.679449 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.679255    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 16:44:36.710797 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.710765    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 16:44:36.710950 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.710914    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.714616 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.713938    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 16:44:36.714616 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.713942    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 16:44:36.714616 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.713948    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 16:44:36.714616 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.714194    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 16:44:36.714616 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.714297    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 16:44:36.714616 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.714388    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 16:44:36.717471 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.716412    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 16:44:36.717471 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.716673    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 16:44:36.717471 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.716910    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 16:44:36.717471 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.717153    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-erla2m3lolpcg\""
Sep 11 16:44:36.717471 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.717340    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 16:44:36.717779 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.717629    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 16:44:36.717779 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.717652    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 16:44:36.717779 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.717697    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 16:44:36.719408 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.717956    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 16:44:36.719408 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.718225    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p66tp\""
Sep 11 16:44:36.720036 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.720012    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 16:44:36.832172 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832038    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832172 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832113    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832172 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832167    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832517 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832517 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832228    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832517 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832247    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832517 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832285    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832517 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832312    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832517 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832415    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832517 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832504    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832531    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832556    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832634    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-config-out\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832660    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-web-config\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832692    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-config\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832721    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfzh7\" (UniqueName: \"kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-kube-api-access-qfzh7\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832749    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832778    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.832809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.832801    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934006    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934098    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934123    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934159    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934182    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934210    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934250    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934274    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934297    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934325    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934346    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-config-out\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934369    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-web-config\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934395    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-config\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934909 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfzh7\" (UniqueName: \"kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-kube-api-access-qfzh7\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934909 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934457    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934909 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934491    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934909 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934516    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934909 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934558    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.934909 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934592    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.935237 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.934960    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.935237 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.935098    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.935395 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.935349    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.937424 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.937394    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.938432 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.938404    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.939207 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.939155    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.939683 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.939655    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.939907 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.939859    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.940043 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.940001    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.940413 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.940381    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.940525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.940498    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.941064 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.941043    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.941424 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.941400    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.943328 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.943306    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.945151 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.945105    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfzh7\" (UniqueName: \"kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-kube-api-access-qfzh7\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.945247 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.945153    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-web-config\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.945247 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.945205    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.945247 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.945218    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-config-out\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.945566 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.945541    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:36.947061 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:36.947039    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-config\") pod \"prometheus-k8s-0\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:37.027057 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:37.027018    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:44:39.706818 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:39.706783    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:39.785442 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:39.785405    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:40.610571 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:40.610535    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:40.665237 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:40.665204    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:40.896424 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:40.896372    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:40.995775 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:40.995731    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:41.710777 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:41.710742    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d5cc66cb-92qfs"]
Sep 11 16:44:43.165200 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:43.165103    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:43.223086 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:43.223035    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:43.307291 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:43.307247    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:43.366598 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:43.366560    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:44.311158 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:44.311122    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"]
Sep 11 16:44:44.328406 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:44.328376    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d5cc66cb-92qfs"]
Sep 11 16:44:44.332680 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:44.332657    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 16:44:44.620740 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:44.620705    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d26f5d_57c5_439d_b1a0_5128ecb947af.slice/crio-5cbe56aba55fbe3cbc4dc3bb31d7a2da66b9618e31e5a22ceb7a0bcb8f8cf736 WatchSource:0}: Error finding container 5cbe56aba55fbe3cbc4dc3bb31d7a2da66b9618e31e5a22ceb7a0bcb8f8cf736: Status 404 returned error can't find the container with id 5cbe56aba55fbe3cbc4dc3bb31d7a2da66b9618e31e5a22ceb7a0bcb8f8cf736
Sep 11 16:44:44.621608 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:44.621583    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc0b8a1a_a005_4d8d_a407_a93598edfc6d.slice/crio-1d7a9161ca8ed10676bda243dc59d186b8027df91dc38bf2a6b32781600be176 WatchSource:0}: Error finding container 1d7a9161ca8ed10676bda243dc59d186b8027df91dc38bf2a6b32781600be176: Status 404 returned error can't find the container with id 1d7a9161ca8ed10676bda243dc59d186b8027df91dc38bf2a6b32781600be176
Sep 11 16:44:44.623860 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:44:44.623707    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce28c939_7365_44a2_a4e4_634e634acb29.slice/crio-24597c6d4b40be915eed0124ea0b41f5c11917277664d8f89110c5e1277c31cf WatchSource:0}: Error finding container 24597c6d4b40be915eed0124ea0b41f5c11917277664d8f89110c5e1277c31cf: Status 404 returned error can't find the container with id 24597c6d4b40be915eed0124ea0b41f5c11917277664d8f89110c5e1277c31cf
Sep 11 16:44:45.509301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.509269    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8f7d7fc8c-4pwmv"
Sep 11 16:44:45.627798 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.626565    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-gvcnc" event={"ID":"2ca0adb8-a1e2-42fe-b506-67c40d3de813","Type":"ContainerStarted","Data":"f5b753b7cf7276af17d974e06e6dd09884c29ff5480ed607132034b82e4f814a"}
Sep 11 16:44:45.627798 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.627129    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-gvcnc"
Sep 11 16:44:45.630964 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.630940    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7p27r" event={"ID":"0c446eb1-a854-404f-9106-98d00a6de58c","Type":"ContainerStarted","Data":"c4ed0a86229260d25773f1ff05a1fab0cb0b4304c4befb8297883e74b4692914"}
Sep 11 16:44:45.631098 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.630990    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:44:45.634548 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.634528    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8thxd" event={"ID":"1189ee6b-cf59-424e-82bc-e1b7d8345eaf","Type":"ContainerStarted","Data":"c37d0fb663edf1761715328ef7c773d8f9ce3414cf92d4181041ff195e2ba2f9"}
Sep 11 16:44:45.634647 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.634612    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8thxd" event={"ID":"1189ee6b-cf59-424e-82bc-e1b7d8345eaf","Type":"ContainerStarted","Data":"c1a25eda7a62421d13d60e7d23df7dd8a31bdd79f3a41b98751b8cc6d6395aa5"}
Sep 11 16:44:45.639521 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.639486    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d5cc66cb-92qfs" event={"ID":"ce28c939-7365-44a2-a4e4-634e634acb29","Type":"ContainerStarted","Data":"6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab"}
Sep 11 16:44:45.639675 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.639660    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d5cc66cb-92qfs" event={"ID":"ce28c939-7365-44a2-a4e4-634e634acb29","Type":"ContainerStarted","Data":"24597c6d4b40be915eed0124ea0b41f5c11917277664d8f89110c5e1277c31cf"}
Sep 11 16:44:45.639878 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.639863    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-gvcnc"
Sep 11 16:44:45.642060 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.642032    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b6fm6" event={"ID":"a5e339d2-e61e-4cf6-8f35-aafb644b4136","Type":"ContainerStarted","Data":"fb995c23ec23c9fd069cd49680f33781250076b3450763589e1aa2806c2e100d"}
Sep 11 16:44:45.647697 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.647528    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-gvcnc" podStartSLOduration=2.467862497 podStartE2EDuration="24.647510407s" totalImagesPullingTime="22.17964791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:45.647505311 +0000 UTC m=+82.054723992" watchObservedRunningTime="2026-09-11 16:44:45.647510407 +0000 UTC m=+82.054729056"
Sep 11 16:44:45.648828 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.648642    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx" event={"ID":"fd73d7c4-ea76-4820-8c34-c74f014fa722","Type":"ContainerStarted","Data":"2d806a17c3f2f67ea095cac4e7a99114803227bbc0fd17b4485c22ba8c76c64c"}
Sep 11 16:44:45.648828 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.648715    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx" event={"ID":"fd73d7c4-ea76-4820-8c34-c74f014fa722","Type":"ContainerStarted","Data":"059169d92c21d2811991978b37b9b64815a2d77eeafedaae41686ccaa5978d9b"}
Sep 11 16:44:45.648828 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.648759    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx" event={"ID":"fd73d7c4-ea76-4820-8c34-c74f014fa722","Type":"ContainerStarted","Data":"bd2283f2f44c7bbdab05704cd4fdcf3a1fa13329eb0f65773393ce103ff3b61a"}
Sep 11 16:44:45.651588 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.651568    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7" event={"ID":"bc0b8a1a-a005-4d8d-a407-a93598edfc6d","Type":"ContainerStarted","Data":"1d7a9161ca8ed10676bda243dc59d186b8027df91dc38bf2a6b32781600be176"}
Sep 11 16:44:45.654490 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.654415    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerStarted","Data":"5cbe56aba55fbe3cbc4dc3bb31d7a2da66b9618e31e5a22ceb7a0bcb8f8cf736"}
Sep 11 16:44:45.670342 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.670122    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8thxd" podStartSLOduration=13.998988052 podStartE2EDuration="15.670102297s" totalImagesPullingTime="1.589053353s" totalInitContainerRuntime="82.060892ms" isStatefulPod=true podCreationTimestamp="2026-09-11 16:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:45.668174634 +0000 UTC m=+82.075393318" watchObservedRunningTime="2026-09-11 16:44:45.670102297 +0000 UTC m=+82.077320946"
Sep 11 16:44:45.673126 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.672836    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:45.692342 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.692221    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85d5cc66cb-92qfs" podStartSLOduration=9.692205209 podStartE2EDuration="9.692205209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:45.690939657 +0000 UTC m=+82.098158307" watchObservedRunningTime="2026-09-11 16:44:45.692205209 +0000 UTC m=+82.099423860"
Sep 11 16:44:45.747836 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.747765    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7p27r" podStartSLOduration=68.223933585 podStartE2EDuration="1m21.747728017s" totalImagesPullingTime="13.523794432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:45.745788401 +0000 UTC m=+82.153007062" watchObservedRunningTime="2026-09-11 16:44:45.747728017 +0000 UTC m=+82.154946665"
Sep 11 16:44:45.749750 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.749701    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b6fm6" podStartSLOduration=67.327035247 podStartE2EDuration="1m21.749685544s" totalImagesPullingTime="14.422650297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:45.719707379 +0000 UTC m=+82.126926060" watchObservedRunningTime="2026-09-11 16:44:45.749685544 +0000 UTC m=+82.156904194"
Sep 11 16:44:45.756832 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.756758    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:45.872474 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.872338    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:45.979384 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:45.979343    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:46.660705 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:46.660594    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:44:47.002595 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:47.002514    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:47.099762 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:47.099735    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:47.666460 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:47.666433    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx" event={"ID":"fd73d7c4-ea76-4820-8c34-c74f014fa722","Type":"ContainerStarted","Data":"9ce12ab0e1549a5fb1640ed97ca386f6bf49b26ac33a1ab6a1a81617263209e8"}
Sep 11 16:44:47.668243 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:47.668211    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7" event={"ID":"bc0b8a1a-a005-4d8d-a407-a93598edfc6d","Type":"ContainerStarted","Data":"025ee84ce503d6babf83f75000c6ce315f57f27f9d6a076d959a3f4ab4ba55b2"}
Sep 11 16:44:47.668440 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:47.668392    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"
Sep 11 16:44:47.670169 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:47.670140    2569 generic.go:356] "Generic (PLEG): container finished" podID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerID="a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6" exitCode=0
Sep 11 16:44:47.670313 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:47.670278    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerDied","Data":"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"}
Sep 11 16:44:47.674316 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:47.674295    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7"
Sep 11 16:44:47.697168 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:47.697123    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wxsp7" podStartSLOduration=10.278703774 podStartE2EDuration="12.697103902s" totalImagesPullingTime="2.418400128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:47.696216106 +0000 UTC m=+84.103434779" watchObservedRunningTime="2026-09-11 16:44:47.697103902 +0000 UTC m=+84.104322551"
Sep 11 16:44:48.680740 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:48.680695    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx" event={"ID":"fd73d7c4-ea76-4820-8c34-c74f014fa722","Type":"ContainerStarted","Data":"1ab28d18c2dc096c720c9cdce3288c16ea7f6c099e79575fa3e0e25028dbe539"}
Sep 11 16:44:48.681481 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:48.680787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx" event={"ID":"fd73d7c4-ea76-4820-8c34-c74f014fa722","Type":"ContainerStarted","Data":"280dba1db304f009e6feb08b3b19f04f1af923bedf096088362ffc66032cfee1"}
Sep 11 16:44:48.681481 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:48.681100    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:48.706804 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:48.706750    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx" podStartSLOduration=2.675485367 podStartE2EDuration="16.706730368s" totalImagesPullingTime="14.031245001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:48.705960351 +0000 UTC m=+85.113179026" watchObservedRunningTime="2026-09-11 16:44:48.706730368 +0000 UTC m=+85.113949016"
Sep 11 16:44:49.650606 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:49.650570    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:49.692721 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:49.692691    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7ff974997c-trpkx"
Sep 11 16:44:49.731396 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:49.731356    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:50.015049 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:50.014935    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:50.139506 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:50.139468    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:52.536885 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:52.536851    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fc5df6dcb-5w5hg"]
Sep 11 16:44:52.654632 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:52.654586    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:52.705251 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:52.705221    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerStarted","Data":"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"}
Sep 11 16:44:52.705426 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:52.705298    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerStarted","Data":"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"}
Sep 11 16:44:52.705426 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:52.705327    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerStarted","Data":"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"}
Sep 11 16:44:52.705426 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:52.705350    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerStarted","Data":"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"}
Sep 11 16:44:52.705426 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:52.705374    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerStarted","Data":"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"}
Sep 11 16:44:52.705426 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:52.705403    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerStarted","Data":"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"}
Sep 11 16:44:52.737330 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:52.737220    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=10.258800714 podStartE2EDuration="16.737195432s" totalImagesPullingTime="6.458479209s" totalInitContainerRuntime="19.915509ms" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:52.736360811 +0000 UTC m=+89.143579486" watchObservedRunningTime="2026-09-11 16:44:52.737195432 +0000 UTC m=+89.144414080"
Sep 11 16:44:52.755882 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:52.755847    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:52.812648 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:52.812615    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:54.106258 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:54.106226    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:54.204185 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:54.204150    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:54.261195 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:54.261147    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:44:57.027509 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:44:57.027472    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:02.030998 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:02.030968    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:45:02.131737 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:02.131703    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:45:08.227259 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:08.227223    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:45:10.663408 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:10.663344    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85d5cc66cb-92qfs" podUID="ce28c939-7365-44a2-a4e4-634e634acb29" containerName="console" containerID="cri-o://6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab" gracePeriod=15
Sep 11 16:45:10.945621 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:10.945594    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:45:11.042897 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.042860    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-trusted-ca-bundle\") pod \"ce28c939-7365-44a2-a4e4-634e634acb29\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") "
Sep 11 16:45:11.043069 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.042919    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-console-config\") pod \"ce28c939-7365-44a2-a4e4-634e634acb29\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") "
Sep 11 16:45:11.043069 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.042940    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-service-ca\") pod \"ce28c939-7365-44a2-a4e4-634e634acb29\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") "
Sep 11 16:45:11.043069 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.042969    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-serving-cert\") pod \"ce28c939-7365-44a2-a4e4-634e634acb29\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") "
Sep 11 16:45:11.043069 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.042986    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-oauth-serving-cert\") pod \"ce28c939-7365-44a2-a4e4-634e634acb29\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") "
Sep 11 16:45:11.043069 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.043045    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-oauth-config\") pod \"ce28c939-7365-44a2-a4e4-634e634acb29\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") "
Sep 11 16:45:11.043347 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.043098    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wntgb\" (UniqueName: \"kubernetes.io/projected/ce28c939-7365-44a2-a4e4-634e634acb29-kube-api-access-wntgb\") pod \"ce28c939-7365-44a2-a4e4-634e634acb29\" (UID: \"ce28c939-7365-44a2-a4e4-634e634acb29\") "
Sep 11 16:45:11.043773 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.043581    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ce28c939-7365-44a2-a4e4-634e634acb29" (UID: "ce28c939-7365-44a2-a4e4-634e634acb29"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:11.043773 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.043696    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-service-ca" (OuterVolumeSpecName: "service-ca") pod "ce28c939-7365-44a2-a4e4-634e634acb29" (UID: "ce28c939-7365-44a2-a4e4-634e634acb29"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:11.043773 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.043729    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-console-config" (OuterVolumeSpecName: "console-config") pod "ce28c939-7365-44a2-a4e4-634e634acb29" (UID: "ce28c939-7365-44a2-a4e4-634e634acb29"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:11.044013 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.043982    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ce28c939-7365-44a2-a4e4-634e634acb29" (UID: "ce28c939-7365-44a2-a4e4-634e634acb29"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:11.047566 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.046463    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce28c939-7365-44a2-a4e4-634e634acb29-kube-api-access-wntgb" (OuterVolumeSpecName: "kube-api-access-wntgb") pod "ce28c939-7365-44a2-a4e4-634e634acb29" (UID: "ce28c939-7365-44a2-a4e4-634e634acb29"). InnerVolumeSpecName "kube-api-access-wntgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:45:11.047566 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.046476    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ce28c939-7365-44a2-a4e4-634e634acb29" (UID: "ce28c939-7365-44a2-a4e4-634e634acb29"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:11.050851 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.050828    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ce28c939-7365-44a2-a4e4-634e634acb29" (UID: "ce28c939-7365-44a2-a4e4-634e634acb29"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:11.144026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.143992    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-console-config\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:11.144026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.144021    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:11.144026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.144029    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-serving-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:11.144259 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.144062    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-oauth-serving-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:11.144259 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.144071    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce28c939-7365-44a2-a4e4-634e634acb29-console-oauth-config\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:11.144259 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.144099    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wntgb\" (UniqueName: \"kubernetes.io/projected/ce28c939-7365-44a2-a4e4-634e634acb29-kube-api-access-wntgb\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:11.144259 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.144108    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce28c939-7365-44a2-a4e4-634e634acb29-trusted-ca-bundle\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:11.780201 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.780169    2569 generic.go:356] "Generic (PLEG): container finished" podID="ce28c939-7365-44a2-a4e4-634e634acb29" containerID="6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab" exitCode=0
Sep 11 16:45:11.780619 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.780207    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d5cc66cb-92qfs" event={"ID":"ce28c939-7365-44a2-a4e4-634e634acb29","Type":"ContainerDied","Data":"6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab"}
Sep 11 16:45:11.780619 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.780224    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d5cc66cb-92qfs"
Sep 11 16:45:11.780619 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.780244    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d5cc66cb-92qfs" event={"ID":"ce28c939-7365-44a2-a4e4-634e634acb29","Type":"ContainerDied","Data":"24597c6d4b40be915eed0124ea0b41f5c11917277664d8f89110c5e1277c31cf"}
Sep 11 16:45:11.780619 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.780275    2569 scope.go:117] "RemoveContainer" containerID="6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab"
Sep 11 16:45:11.789224 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.789199    2569 scope.go:117] "RemoveContainer" containerID="6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab"
Sep 11 16:45:11.789511 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:45:11.789491    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab\": container with ID starting with 6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab not found: ID does not exist" containerID="6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab"
Sep 11 16:45:11.789571 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.789519    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab"} err="failed to get container status \"6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab\": rpc error: code = NotFound desc = could not find container \"6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab\": container with ID starting with 6de4365529c4ff3f1347155881b4e23d5bc0af1d5712381df07953a0688422ab not found: ID does not exist"
Sep 11 16:45:11.803318 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.803291    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d5cc66cb-92qfs"]
Sep 11 16:45:11.809103 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:11.809058    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85d5cc66cb-92qfs"]
Sep 11 16:45:12.204207 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:12.204175    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce28c939-7365-44a2-a4e4-634e634acb29" path="/var/lib/kubelet/pods/ce28c939-7365-44a2-a4e4-634e634acb29/volumes"
Sep 11 16:45:16.661225 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:16.661188    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7p27r"
Sep 11 16:45:16.971741 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:16.971662    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:45:17.072882 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.072847    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:45:17.127858 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.127824    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:45:17.559790 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.559747    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6fc5df6dcb-5w5hg" podUID="b96801ce-a64a-4b22-831a-fc7ad552c96c" containerName="console" containerID="cri-o://2b6442564c7713e7cc380b6a085bbf3bd637961da4efc16718bf0963af78434e" gracePeriod=15
Sep 11 16:45:17.808417 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.808390    2569 generic.go:356] "Generic (PLEG): container finished" podID="b96801ce-a64a-4b22-831a-fc7ad552c96c" containerID="2b6442564c7713e7cc380b6a085bbf3bd637961da4efc16718bf0963af78434e" exitCode=0
Sep 11 16:45:17.808703 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.808440    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc5df6dcb-5w5hg" event={"ID":"b96801ce-a64a-4b22-831a-fc7ad552c96c","Type":"ContainerDied","Data":"2b6442564c7713e7cc380b6a085bbf3bd637961da4efc16718bf0963af78434e"}
Sep 11 16:45:17.808703 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.808481    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc5df6dcb-5w5hg" event={"ID":"b96801ce-a64a-4b22-831a-fc7ad552c96c","Type":"ContainerDied","Data":"f28b17b32c87e0443281bb0562d93e827b31d0e80362618f7a6e106a2b264f7c"}
Sep 11 16:45:17.808703 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.808497    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f28b17b32c87e0443281bb0562d93e827b31d0e80362618f7a6e106a2b264f7c"
Sep 11 16:45:17.841615 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.841534    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:45:17.901121 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.901059    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-service-ca\") pod \"b96801ce-a64a-4b22-831a-fc7ad552c96c\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") "
Sep 11 16:45:17.901317 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.901144    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-oauth-config\") pod \"b96801ce-a64a-4b22-831a-fc7ad552c96c\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") "
Sep 11 16:45:17.901317 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.901184    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-oauth-serving-cert\") pod \"b96801ce-a64a-4b22-831a-fc7ad552c96c\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") "
Sep 11 16:45:17.901317 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.901209    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cssx5\" (UniqueName: \"kubernetes.io/projected/b96801ce-a64a-4b22-831a-fc7ad552c96c-kube-api-access-cssx5\") pod \"b96801ce-a64a-4b22-831a-fc7ad552c96c\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") "
Sep 11 16:45:17.901317 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.901276    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-config\") pod \"b96801ce-a64a-4b22-831a-fc7ad552c96c\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") "
Sep 11 16:45:17.901317 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.901317    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-serving-cert\") pod \"b96801ce-a64a-4b22-831a-fc7ad552c96c\" (UID: \"b96801ce-a64a-4b22-831a-fc7ad552c96c\") "
Sep 11 16:45:17.901652 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.901519    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-service-ca" (OuterVolumeSpecName: "service-ca") pod "b96801ce-a64a-4b22-831a-fc7ad552c96c" (UID: "b96801ce-a64a-4b22-831a-fc7ad552c96c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:17.901652 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.901630    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b96801ce-a64a-4b22-831a-fc7ad552c96c" (UID: "b96801ce-a64a-4b22-831a-fc7ad552c96c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:17.901759 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.901703    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:17.901759 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.901739    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-config" (OuterVolumeSpecName: "console-config") pod "b96801ce-a64a-4b22-831a-fc7ad552c96c" (UID: "b96801ce-a64a-4b22-831a-fc7ad552c96c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:17.903515 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.903492    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b96801ce-a64a-4b22-831a-fc7ad552c96c" (UID: "b96801ce-a64a-4b22-831a-fc7ad552c96c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:17.903644 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.903609    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b96801ce-a64a-4b22-831a-fc7ad552c96c" (UID: "b96801ce-a64a-4b22-831a-fc7ad552c96c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:17.903723 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:17.903705    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b96801ce-a64a-4b22-831a-fc7ad552c96c-kube-api-access-cssx5" (OuterVolumeSpecName: "kube-api-access-cssx5") pod "b96801ce-a64a-4b22-831a-fc7ad552c96c" (UID: "b96801ce-a64a-4b22-831a-fc7ad552c96c"). InnerVolumeSpecName "kube-api-access-cssx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:45:18.002488 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:18.002449    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-oauth-config\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:18.002488 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:18.002490    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-oauth-serving-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:18.002683 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:18.002506    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cssx5\" (UniqueName: \"kubernetes.io/projected/b96801ce-a64a-4b22-831a-fc7ad552c96c-kube-api-access-cssx5\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:18.002683 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:18.002521    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-config\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:18.002683 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:18.002535    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b96801ce-a64a-4b22-831a-fc7ad552c96c-console-serving-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:18.812384 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:18.812352    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc5df6dcb-5w5hg"
Sep 11 16:45:18.830245 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:18.830215    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fc5df6dcb-5w5hg"]
Sep 11 16:45:18.834325 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:18.834300    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fc5df6dcb-5w5hg"]
Sep 11 16:45:20.204218 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:20.204173    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b96801ce-a64a-4b22-831a-fc7ad552c96c" path="/var/lib/kubelet/pods/b96801ce-a64a-4b22-831a-fc7ad552c96c/volumes"
Sep 11 16:45:37.027360 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:37.027327    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:37.109892 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:37.109868    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:37.908604 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:37.908575    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:46.579536 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:46.579500    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:45:46.645683 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:46.645654    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:45:54.664681 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:54.664635    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:45:55.211008 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.210965    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 16:45:55.211452 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.211415    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="prometheus" containerID="cri-o://5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932" gracePeriod=600
Sep 11 16:45:55.211695 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.211429    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646" gracePeriod=600
Sep 11 16:45:55.211811 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.211492    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="config-reloader" containerID="cri-o://1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b" gracePeriod=600
Sep 11 16:45:55.211811 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.211478    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy" containerID="cri-o://76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776" gracePeriod=600
Sep 11 16:45:55.211920 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.211477    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy-web" containerID="cri-o://8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f" gracePeriod=600
Sep 11 16:45:55.211920 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.211496    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="thanos-sidecar" containerID="cri-o://a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c" gracePeriod=600
Sep 11 16:45:55.665385 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.665353    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:55.715061 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715026    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-metrics-client-certs\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715261 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715088    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-kube-rbac-proxy\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715261 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715123    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-thanos-prometheus-http-client-file\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715261 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715149    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-db\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715261 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715178    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfzh7\" (UniqueName: \"kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-kube-api-access-qfzh7\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715261 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715203    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-0\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715261 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715238    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-1\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715263    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-trusted-ca-bundle\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715315    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-config-out\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715340    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715372    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715397    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-tls-assets\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715444    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-kubelet-serving-ca-bundle\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715468    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-tls\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715496    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-web-config\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715526    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-grpc-tls\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715552    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-serving-certs-ca-bundle\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.715579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715578    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-2\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.716112 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715627    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-metrics-client-ca\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.716112 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715634    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:55.716112 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715661    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-config\") pod \"76d26f5d-57c5-439d-b1a0-5128ecb947af\" (UID: \"76d26f5d-57c5-439d-b1a0-5128ecb947af\") "
Sep 11 16:45:55.716112 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.715928    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.716620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.716344    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:55.717607 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.717569    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:55.718365 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.718052    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:55.718365 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.718137    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:55.718365 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.718287    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:55.719024 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.718751    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:55.719647 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.719608    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:55.720799 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.720771    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:55.720952 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.720932    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:55.721131 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.721059    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-config" (OuterVolumeSpecName: "config") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:55.721549 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.721423    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:55.721549 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.721521    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:55.721708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.721521    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-kube-api-access-qfzh7" (OuterVolumeSpecName: "kube-api-access-qfzh7") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "kube-api-access-qfzh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:45:55.721708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.721605    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:55.722825 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.722508    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:55.722825 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.722652    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:55.722825 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.722738    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:45:55.723189 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.723170    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-config-out" (OuterVolumeSpecName: "config-out") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:55.736022 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.735982    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-web-config" (OuterVolumeSpecName: "web-config") pod "76d26f5d-57c5-439d-b1a0-5128ecb947af" (UID: "76d26f5d-57c5-439d-b1a0-5128ecb947af"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:55.817277 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817185    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfzh7\" (UniqueName: \"kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-kube-api-access-qfzh7\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817277 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817219    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817277 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817229    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817277 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817239    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-config-out\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817277 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817249    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817277 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817260    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817277 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817268    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/76d26f5d-57c5-439d-b1a0-5128ecb947af-tls-assets\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817277 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817277    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817277 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817285    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817294    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-web-config\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817304    2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-grpc-tls\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817312    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817320    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817329    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76d26f5d-57c5-439d-b1a0-5128ecb947af-configmap-metrics-client-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817337    2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-config\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817345    2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-metrics-client-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817354    2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-secret-kube-rbac-proxy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817363    2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/76d26f5d-57c5-439d-b1a0-5128ecb947af-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.817658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.817373    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/76d26f5d-57c5-439d-b1a0-5128ecb947af-prometheus-k8s-db\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:45:55.964390 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964356    2569 generic.go:356] "Generic (PLEG): container finished" podID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerID="1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646" exitCode=0
Sep 11 16:45:55.964390 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964382    2569 generic.go:356] "Generic (PLEG): container finished" podID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerID="76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776" exitCode=0
Sep 11 16:45:55.964390 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964389    2569 generic.go:356] "Generic (PLEG): container finished" podID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerID="8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f" exitCode=0
Sep 11 16:45:55.964390 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964397    2569 generic.go:356] "Generic (PLEG): container finished" podID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerID="a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c" exitCode=0
Sep 11 16:45:55.964627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964402    2569 generic.go:356] "Generic (PLEG): container finished" podID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerID="1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b" exitCode=0
Sep 11 16:45:55.964627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964408    2569 generic.go:356] "Generic (PLEG): container finished" podID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerID="5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932" exitCode=0
Sep 11 16:45:55.964627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964433    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:55.964627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964441    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerDied","Data":"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"}
Sep 11 16:45:55.964627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964511    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerDied","Data":"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"}
Sep 11 16:45:55.964627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964540    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerDied","Data":"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"}
Sep 11 16:45:55.964627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964564    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerDied","Data":"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"}
Sep 11 16:45:55.964627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964578    2569 scope.go:117] "RemoveContainer" containerID="1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"
Sep 11 16:45:55.964627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964588    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerDied","Data":"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"}
Sep 11 16:45:55.964627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964614    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerDied","Data":"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"}
Sep 11 16:45:55.965228 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.964638    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76d26f5d-57c5-439d-b1a0-5128ecb947af","Type":"ContainerDied","Data":"5cbe56aba55fbe3cbc4dc3bb31d7a2da66b9618e31e5a22ceb7a0bcb8f8cf736"}
Sep 11 16:45:55.973871 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.973849    2569 scope.go:117] "RemoveContainer" containerID="76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"
Sep 11 16:45:55.981293 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.981275    2569 scope.go:117] "RemoveContainer" containerID="8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"
Sep 11 16:45:55.988433 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.988410    2569 scope.go:117] "RemoveContainer" containerID="a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"
Sep 11 16:45:55.996136 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.996109    2569 scope.go:117] "RemoveContainer" containerID="1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"
Sep 11 16:45:55.996838 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:55.996813    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 16:45:56.004135 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.004106    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 16:45:56.004500 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.004475    2569 scope.go:117] "RemoveContainer" containerID="5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"
Sep 11 16:45:56.012391 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.012370    2569 scope.go:117] "RemoveContainer" containerID="a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"
Sep 11 16:45:56.019812 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.019794    2569 scope.go:117] "RemoveContainer" containerID="1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"
Sep 11 16:45:56.020175 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:45:56.020156    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": container with ID starting with 1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646 not found: ID does not exist" containerID="1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"
Sep 11 16:45:56.020222 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.020187    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"} err="failed to get container status \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": rpc error: code = NotFound desc = could not find container \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": container with ID starting with 1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646 not found: ID does not exist"
Sep 11 16:45:56.020222 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.020208    2569 scope.go:117] "RemoveContainer" containerID="76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"
Sep 11 16:45:56.020516 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:45:56.020496    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": container with ID starting with 76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776 not found: ID does not exist" containerID="76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"
Sep 11 16:45:56.020551 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.020523    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"} err="failed to get container status \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": rpc error: code = NotFound desc = could not find container \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": container with ID starting with 76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776 not found: ID does not exist"
Sep 11 16:45:56.020551 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.020540    2569 scope.go:117] "RemoveContainer" containerID="8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"
Sep 11 16:45:56.020805 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:45:56.020788    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": container with ID starting with 8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f not found: ID does not exist" containerID="8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"
Sep 11 16:45:56.020840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.020812    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"} err="failed to get container status \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": rpc error: code = NotFound desc = could not find container \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": container with ID starting with 8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f not found: ID does not exist"
Sep 11 16:45:56.020840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.020828    2569 scope.go:117] "RemoveContainer" containerID="a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"
Sep 11 16:45:56.021082 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:45:56.021057    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": container with ID starting with a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c not found: ID does not exist" containerID="a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"
Sep 11 16:45:56.021136 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.021102    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"} err="failed to get container status \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": rpc error: code = NotFound desc = could not find container \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": container with ID starting with a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c not found: ID does not exist"
Sep 11 16:45:56.021136 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.021128    2569 scope.go:117] "RemoveContainer" containerID="1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"
Sep 11 16:45:56.021386 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:45:56.021366    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": container with ID starting with 1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b not found: ID does not exist" containerID="1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"
Sep 11 16:45:56.021431 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.021394    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"} err="failed to get container status \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": rpc error: code = NotFound desc = could not find container \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": container with ID starting with 1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b not found: ID does not exist"
Sep 11 16:45:56.021431 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.021413    2569 scope.go:117] "RemoveContainer" containerID="5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"
Sep 11 16:45:56.021662 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:45:56.021646    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": container with ID starting with 5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932 not found: ID does not exist" containerID="5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"
Sep 11 16:45:56.021701 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.021665    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"} err="failed to get container status \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": rpc error: code = NotFound desc = could not find container \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": container with ID starting with 5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932 not found: ID does not exist"
Sep 11 16:45:56.021701 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.021680    2569 scope.go:117] "RemoveContainer" containerID="a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"
Sep 11 16:45:56.021922 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:45:56.021905    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": container with ID starting with a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6 not found: ID does not exist" containerID="a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"
Sep 11 16:45:56.021983 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.021930    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"} err="failed to get container status \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": rpc error: code = NotFound desc = could not find container \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": container with ID starting with a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6 not found: ID does not exist"
Sep 11 16:45:56.021983 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.021951    2569 scope.go:117] "RemoveContainer" containerID="1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"
Sep 11 16:45:56.022222 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.022204    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"} err="failed to get container status \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": rpc error: code = NotFound desc = could not find container \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": container with ID starting with 1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646 not found: ID does not exist"
Sep 11 16:45:56.022275 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.022223    2569 scope.go:117] "RemoveContainer" containerID="76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"
Sep 11 16:45:56.022435 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.022417    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"} err="failed to get container status \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": rpc error: code = NotFound desc = could not find container \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": container with ID starting with 76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776 not found: ID does not exist"
Sep 11 16:45:56.022477 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.022435    2569 scope.go:117] "RemoveContainer" containerID="8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"
Sep 11 16:45:56.022676 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.022655    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"} err="failed to get container status \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": rpc error: code = NotFound desc = could not find container \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": container with ID starting with 8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f not found: ID does not exist"
Sep 11 16:45:56.022750 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.022677    2569 scope.go:117] "RemoveContainer" containerID="a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"
Sep 11 16:45:56.022936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.022918    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"} err="failed to get container status \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": rpc error: code = NotFound desc = could not find container \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": container with ID starting with a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c not found: ID does not exist"
Sep 11 16:45:56.022973 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.022937    2569 scope.go:117] "RemoveContainer" containerID="1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"
Sep 11 16:45:56.023177 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.023158    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"} err="failed to get container status \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": rpc error: code = NotFound desc = could not find container \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": container with ID starting with 1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b not found: ID does not exist"
Sep 11 16:45:56.023257 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.023178    2569 scope.go:117] "RemoveContainer" containerID="5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"
Sep 11 16:45:56.023406 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.023389    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"} err="failed to get container status \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": rpc error: code = NotFound desc = could not find container \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": container with ID starting with 5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932 not found: ID does not exist"
Sep 11 16:45:56.023460 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.023406    2569 scope.go:117] "RemoveContainer" containerID="a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"
Sep 11 16:45:56.023675 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.023658    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"} err="failed to get container status \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": rpc error: code = NotFound desc = could not find container \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": container with ID starting with a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6 not found: ID does not exist"
Sep 11 16:45:56.023675 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.023675    2569 scope.go:117] "RemoveContainer" containerID="1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"
Sep 11 16:45:56.023899 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.023881    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"} err="failed to get container status \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": rpc error: code = NotFound desc = could not find container \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": container with ID starting with 1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646 not found: ID does not exist"
Sep 11 16:45:56.023955 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.023900    2569 scope.go:117] "RemoveContainer" containerID="76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"
Sep 11 16:45:56.024156 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.024138    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"} err="failed to get container status \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": rpc error: code = NotFound desc = could not find container \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": container with ID starting with 76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776 not found: ID does not exist"
Sep 11 16:45:56.024156 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.024156    2569 scope.go:117] "RemoveContainer" containerID="8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"
Sep 11 16:45:56.024386 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.024363    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"} err="failed to get container status \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": rpc error: code = NotFound desc = could not find container \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": container with ID starting with 8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f not found: ID does not exist"
Sep 11 16:45:56.024428 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.024387    2569 scope.go:117] "RemoveContainer" containerID="a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"
Sep 11 16:45:56.024593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.024578    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"} err="failed to get container status \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": rpc error: code = NotFound desc = could not find container \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": container with ID starting with a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c not found: ID does not exist"
Sep 11 16:45:56.024638 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.024593    2569 scope.go:117] "RemoveContainer" containerID="1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"
Sep 11 16:45:56.024832 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.024815    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"} err="failed to get container status \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": rpc error: code = NotFound desc = could not find container \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": container with ID starting with 1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b not found: ID does not exist"
Sep 11 16:45:56.024870 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.024832    2569 scope.go:117] "RemoveContainer" containerID="5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"
Sep 11 16:45:56.025050 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.025036    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"} err="failed to get container status \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": rpc error: code = NotFound desc = could not find container \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": container with ID starting with 5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932 not found: ID does not exist"
Sep 11 16:45:56.025114 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.025050    2569 scope.go:117] "RemoveContainer" containerID="a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"
Sep 11 16:45:56.025305 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.025282    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"} err="failed to get container status \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": rpc error: code = NotFound desc = could not find container \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": container with ID starting with a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6 not found: ID does not exist"
Sep 11 16:45:56.025363 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.025305    2569 scope.go:117] "RemoveContainer" containerID="1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"
Sep 11 16:45:56.025537 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.025520    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"} err="failed to get container status \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": rpc error: code = NotFound desc = could not find container \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": container with ID starting with 1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646 not found: ID does not exist"
Sep 11 16:45:56.025578 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.025539    2569 scope.go:117] "RemoveContainer" containerID="76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"
Sep 11 16:45:56.025779 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.025763    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"} err="failed to get container status \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": rpc error: code = NotFound desc = could not find container \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": container with ID starting with 76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776 not found: ID does not exist"
Sep 11 16:45:56.025779 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.025777    2569 scope.go:117] "RemoveContainer" containerID="8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"
Sep 11 16:45:56.026006 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.025987    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"} err="failed to get container status \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": rpc error: code = NotFound desc = could not find container \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": container with ID starting with 8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f not found: ID does not exist"
Sep 11 16:45:56.026051 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.026007    2569 scope.go:117] "RemoveContainer" containerID="a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"
Sep 11 16:45:56.026263 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.026246    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"} err="failed to get container status \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": rpc error: code = NotFound desc = could not find container \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": container with ID starting with a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c not found: ID does not exist"
Sep 11 16:45:56.026322 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.026270    2569 scope.go:117] "RemoveContainer" containerID="1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"
Sep 11 16:45:56.026515 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.026494    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"} err="failed to get container status \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": rpc error: code = NotFound desc = could not find container \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": container with ID starting with 1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b not found: ID does not exist"
Sep 11 16:45:56.026560 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.026515    2569 scope.go:117] "RemoveContainer" containerID="5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"
Sep 11 16:45:56.026759 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.026738    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"} err="failed to get container status \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": rpc error: code = NotFound desc = could not find container \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": container with ID starting with 5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932 not found: ID does not exist"
Sep 11 16:45:56.026805 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.026759    2569 scope.go:117] "RemoveContainer" containerID="a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"
Sep 11 16:45:56.026977 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.026960    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"} err="failed to get container status \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": rpc error: code = NotFound desc = could not find container \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": container with ID starting with a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6 not found: ID does not exist"
Sep 11 16:45:56.027021 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.026978    2569 scope.go:117] "RemoveContainer" containerID="1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"
Sep 11 16:45:56.027214 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.027199    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"} err="failed to get container status \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": rpc error: code = NotFound desc = could not find container \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": container with ID starting with 1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646 not found: ID does not exist"
Sep 11 16:45:56.027270 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.027214    2569 scope.go:117] "RemoveContainer" containerID="76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"
Sep 11 16:45:56.027435 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.027418    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"} err="failed to get container status \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": rpc error: code = NotFound desc = could not find container \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": container with ID starting with 76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776 not found: ID does not exist"
Sep 11 16:45:56.027478 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.027436    2569 scope.go:117] "RemoveContainer" containerID="8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"
Sep 11 16:45:56.027646 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.027632    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"} err="failed to get container status \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": rpc error: code = NotFound desc = could not find container \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": container with ID starting with 8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f not found: ID does not exist"
Sep 11 16:45:56.027696 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.027646    2569 scope.go:117] "RemoveContainer" containerID="a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"
Sep 11 16:45:56.027856 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.027838    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"} err="failed to get container status \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": rpc error: code = NotFound desc = could not find container \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": container with ID starting with a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c not found: ID does not exist"
Sep 11 16:45:56.027899 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.027856    2569 scope.go:117] "RemoveContainer" containerID="1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"
Sep 11 16:45:56.028090 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.028051    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"} err="failed to get container status \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": rpc error: code = NotFound desc = could not find container \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": container with ID starting with 1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b not found: ID does not exist"
Sep 11 16:45:56.028090 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.028087    2569 scope.go:117] "RemoveContainer" containerID="5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"
Sep 11 16:45:56.028317 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.028303    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"} err="failed to get container status \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": rpc error: code = NotFound desc = could not find container \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": container with ID starting with 5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932 not found: ID does not exist"
Sep 11 16:45:56.028364 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.028316    2569 scope.go:117] "RemoveContainer" containerID="a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"
Sep 11 16:45:56.028537 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.028521    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"} err="failed to get container status \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": rpc error: code = NotFound desc = could not find container \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": container with ID starting with a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6 not found: ID does not exist"
Sep 11 16:45:56.028587 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.028537    2569 scope.go:117] "RemoveContainer" containerID="1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"
Sep 11 16:45:56.028774 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.028759    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646"} err="failed to get container status \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": rpc error: code = NotFound desc = could not find container \"1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646\": container with ID starting with 1594f948a675413bf3e054c9f65078b6871f21af5aabd3fb0e0e4c2c434ec646 not found: ID does not exist"
Sep 11 16:45:56.028774 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.028772    2569 scope.go:117] "RemoveContainer" containerID="76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"
Sep 11 16:45:56.028995 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.028980    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776"} err="failed to get container status \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": rpc error: code = NotFound desc = could not find container \"76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776\": container with ID starting with 76e98aa65c71eae85a5b80e26908e461c4f9d28edeabc2336e4bf7fe68647776 not found: ID does not exist"
Sep 11 16:45:56.029034 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.028995    2569 scope.go:117] "RemoveContainer" containerID="8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"
Sep 11 16:45:56.029243 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.029226    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f"} err="failed to get container status \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": rpc error: code = NotFound desc = could not find container \"8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f\": container with ID starting with 8d67d74f7def750703fa52900995823785de35dbb26fd5a3d705cccef5514b4f not found: ID does not exist"
Sep 11 16:45:56.029296 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.029245    2569 scope.go:117] "RemoveContainer" containerID="a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"
Sep 11 16:45:56.029493 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.029476    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c"} err="failed to get container status \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": rpc error: code = NotFound desc = could not find container \"a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c\": container with ID starting with a2d25c75ac545f19b80c1b685810630434b172c7a180ba596451c3596d7aa16c not found: ID does not exist"
Sep 11 16:45:56.029543 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.029493    2569 scope.go:117] "RemoveContainer" containerID="1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"
Sep 11 16:45:56.029711 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.029692    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b"} err="failed to get container status \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": rpc error: code = NotFound desc = could not find container \"1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b\": container with ID starting with 1bfbcd00d4d823222cae09e34619512be6d42c1b4198b20722027345b1e5f31b not found: ID does not exist"
Sep 11 16:45:56.029758 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.029714    2569 scope.go:117] "RemoveContainer" containerID="5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"
Sep 11 16:45:56.029939 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.029923    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932"} err="failed to get container status \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": rpc error: code = NotFound desc = could not find container \"5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932\": container with ID starting with 5dc9efdefcce31d28772fab3d8788e406d65e3a53a43b4bf409f3d05c4c73932 not found: ID does not exist"
Sep 11 16:45:56.029991 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.029940    2569 scope.go:117] "RemoveContainer" containerID="a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"
Sep 11 16:45:56.030195 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.030175    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6"} err="failed to get container status \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": rpc error: code = NotFound desc = could not find container \"a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6\": container with ID starting with a75bab5e0fb48e1a033f0f2e4b7bdce5dc4068d5eeba18365665087ea7e9a9b6 not found: ID does not exist"
Sep 11 16:45:56.050146 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050108    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 16:45:56.050402 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050390    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy"
Sep 11 16:45:56.050451 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050404    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy"
Sep 11 16:45:56.050451 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050413    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="init-config-reloader"
Sep 11 16:45:56.050451 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050421    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="init-config-reloader"
Sep 11 16:45:56.050451 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050427    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy-web"
Sep 11 16:45:56.050451 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050432    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy-web"
Sep 11 16:45:56.050451 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050442    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce28c939-7365-44a2-a4e4-634e634acb29" containerName="console"
Sep 11 16:45:56.050451 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050447    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce28c939-7365-44a2-a4e4-634e634acb29" containerName="console"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050455    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="prometheus"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050460    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="prometheus"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050465    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy-thanos"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050471    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy-thanos"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050479    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b96801ce-a64a-4b22-831a-fc7ad552c96c" containerName="console"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050484    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b96801ce-a64a-4b22-831a-fc7ad552c96c" containerName="console"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050491    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="config-reloader"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050495    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="config-reloader"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050502    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="thanos-sidecar"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050507    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="thanos-sidecar"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050546    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy-web"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050553    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy-thanos"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050559    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="prometheus"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050564    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="thanos-sidecar"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050570    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="kube-rbac-proxy"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050578    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce28c939-7365-44a2-a4e4-634e634acb29" containerName="console"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050583    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" containerName="config-reloader"
Sep 11 16:45:56.050672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.050589    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b96801ce-a64a-4b22-831a-fc7ad552c96c" containerName="console"
Sep 11 16:45:56.054196 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.054173    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.057670 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.057645    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 16:45:56.057841 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.057827    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 16:45:56.057934 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.057914    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 16:45:56.058040 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.058024    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 16:45:56.058239 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.058226    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 16:45:56.061667 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.061644    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 16:45:56.061811 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.061769    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 16:45:56.061886 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.061825    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 16:45:56.061950 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.061936    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 16:45:56.062004 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.061961    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 16:45:56.062304 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.062280    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 16:45:56.062414 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.062372    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p66tp\""
Sep 11 16:45:56.062414 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.062370    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-erla2m3lolpcg\""
Sep 11 16:45:56.062414 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.062388    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 16:45:56.062569 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.062499    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 16:45:56.065751 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.065730    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 16:45:56.068747 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.068685    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 16:45:56.087178 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.087045    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 16:45:56.119231 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.119231 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119233    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.119469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119252    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71105af7-a093-4ea9-8711-dd717ec483f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.119469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119267    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.119469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119357    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-config\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.119469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119385    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.119469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119409    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.119469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.119841 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119819    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.119908 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119869    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.119959 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119897    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k2ll\" (UniqueName: \"kubernetes.io/projected/71105af7-a093-4ea9-8711-dd717ec483f8-kube-api-access-5k2ll\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.119959 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119941    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.120053 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119965    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71105af7-a093-4ea9-8711-dd717ec483f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.120053 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.119984    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.120172 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.120143    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.120221 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.120190    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.120276 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.120224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.120276 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.120253    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.120369 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.120287    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.120369 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.120321    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.203566 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.203528    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76d26f5d-57c5-439d-b1a0-5128ecb947af" path="/var/lib/kubelet/pods/76d26f5d-57c5-439d-b1a0-5128ecb947af/volumes"
Sep 11 16:45:56.220883 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.220855    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-config\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221066 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.220889    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221066 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.220915    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221066 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221263 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221095    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221263 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221138    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221263 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221168    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k2ll\" (UniqueName: \"kubernetes.io/projected/71105af7-a093-4ea9-8711-dd717ec483f8-kube-api-access-5k2ll\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221263 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221263 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71105af7-a093-4ea9-8711-dd717ec483f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221263 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221263 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221260    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221279    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221296    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221314    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221337    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221368    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221421    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221448    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221477    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71105af7-a093-4ea9-8711-dd717ec483f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.221593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221500    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.222055 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.221882    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.222422 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.222398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.222552 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.222530    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.223944 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.222899    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.223944 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.223664    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.223944 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.224017    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-config\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.223944 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.224149    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71105af7-a093-4ea9-8711-dd717ec483f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.223944 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.224229    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.223944 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.224306    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.223944 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.224550    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.225034 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.224638    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.225034 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.224809    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.225034 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.224946    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.225742 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.225713    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.226272 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.226251    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.226778 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.226742    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71105af7-a093-4ea9-8711-dd717ec483f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.226879 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.226861    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.226988 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.226968    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71105af7-a093-4ea9-8711-dd717ec483f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.227920 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.227897    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71105af7-a093-4ea9-8711-dd717ec483f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.235308 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.235284    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k2ll\" (UniqueName: \"kubernetes.io/projected/71105af7-a093-4ea9-8711-dd717ec483f8-kube-api-access-5k2ll\") pod \"prometheus-k8s-0\" (UID: \"71105af7-a093-4ea9-8711-dd717ec483f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.364229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.364137    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:45:56.651986 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.651962    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 16:45:56.653987 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:45:56.653957    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71105af7_a093_4ea9_8711_dd717ec483f8.slice/crio-cb8ce68888e1b8b91f00a91328cda7cf13b1c18d451e1ea8f4ffa18a8a7bad2f WatchSource:0}: Error finding container cb8ce68888e1b8b91f00a91328cda7cf13b1c18d451e1ea8f4ffa18a8a7bad2f: Status 404 returned error can't find the container with id cb8ce68888e1b8b91f00a91328cda7cf13b1c18d451e1ea8f4ffa18a8a7bad2f
Sep 11 16:45:56.970563 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.970480    2569 generic.go:356] "Generic (PLEG): container finished" podID="71105af7-a093-4ea9-8711-dd717ec483f8" containerID="4189db1328d9225a6449144e116e948c43218b0203bc74610a57347285c974f2" exitCode=0
Sep 11 16:45:56.970926 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.970572    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71105af7-a093-4ea9-8711-dd717ec483f8","Type":"ContainerDied","Data":"4189db1328d9225a6449144e116e948c43218b0203bc74610a57347285c974f2"}
Sep 11 16:45:56.970926 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:56.970642    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71105af7-a093-4ea9-8711-dd717ec483f8","Type":"ContainerStarted","Data":"cb8ce68888e1b8b91f00a91328cda7cf13b1c18d451e1ea8f4ffa18a8a7bad2f"}
Sep 11 16:45:57.977958 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:57.977922    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71105af7-a093-4ea9-8711-dd717ec483f8","Type":"ContainerStarted","Data":"6e8d54039978b9c9be54681ca23231cc66d58769523c2f28265d276b0c291ec9"}
Sep 11 16:45:57.978409 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:57.977997    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71105af7-a093-4ea9-8711-dd717ec483f8","Type":"ContainerStarted","Data":"1a42de895f653bfd53f2f84946504ae22174754ef89d222c38129c8cfad7beb1"}
Sep 11 16:45:57.978409 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:57.978038    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71105af7-a093-4ea9-8711-dd717ec483f8","Type":"ContainerStarted","Data":"8907ab04080c845355cefe8886bd20a87843d5776f1327c746f2fc9d1acab1a1"}
Sep 11 16:45:57.978409 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:57.978098    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71105af7-a093-4ea9-8711-dd717ec483f8","Type":"ContainerStarted","Data":"5c896a88d580e2d564cee1dc7505f2c25845d6e236928b31e6c373476d45b72f"}
Sep 11 16:45:57.978409 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:57.978141    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71105af7-a093-4ea9-8711-dd717ec483f8","Type":"ContainerStarted","Data":"2e279d1d46405c3b57afb96bc77b5b9f7827b5779a3f514c411f6d7d97e28310"}
Sep 11 16:45:57.978409 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:57.978175    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71105af7-a093-4ea9-8711-dd717ec483f8","Type":"ContainerStarted","Data":"c33809a6c2c490ced2f52ac2cf0e0c584ef33a692086f2910aa95bea93fa9acf"}
Sep 11 16:45:58.031158 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:58.031091    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.013780997 podStartE2EDuration="2.031060109s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.279112ms" isStatefulPod=false podCreationTimestamp="2026-09-11 16:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:45:58.029460828 +0000 UTC m=+154.436679476" watchObservedRunningTime="2026-09-11 16:45:58.031060109 +0000 UTC m=+154.438278807"
Sep 11 16:45:58.541378 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:58.541341    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:45:58.622208 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:45:58.622175    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:01.365116 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:01.365057    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:46:02.010405 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:02.010357    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:02.113162 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:02.113108    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:02.197544 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:02.197512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:14.289097 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:14.289040    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:14.353199 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:14.353168    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:25.034468 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:25.034430    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:25.138725 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:25.138693    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:25.244674 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:25.244645    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:33.903829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:33.903796    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:34.009238 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:34.009208    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:34.138269 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:34.138234    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:46.953394 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:46.953359    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:47.072201 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:47.072162    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:56.054001 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:56.053964    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:56.365145 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:56.365036    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:46:56.381050 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:56.381023    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:46:57.064467 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:57.064433    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:57.224894 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:57.224867    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 16:46:58.079936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:58.079903    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:59.850521 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:59.850482    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:46:59.952352 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:46:59.952317    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:01.456262 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:01.456229    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:01.597984 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:01.597946    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:02.108585 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:02.108554    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:02.198469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:02.198438    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:04.427811 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:04.427779    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:04.731580 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:04.731496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:06.870305 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:06.870271    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:06.971443 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:06.971411    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:12.095878 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:12.095833    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:13.938798 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:13.938765    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:14.025425 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:14.025394    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:15.175425 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.175393    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6"]
Sep 11 16:47:15.177719 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.177703    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6"
Sep 11 16:47:15.179719 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.179695    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 16:47:15.180020 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.180003    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:47:15.180121 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.180028    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-6kz2x\""
Sep 11 16:47:15.188848 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.188828    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6"]
Sep 11 16:47:15.269979 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.269942    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c792a287-5a78-4c7b-a6db-a4893b658fc0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cv5g6\" (UID: \"c792a287-5a78-4c7b-a6db-a4893b658fc0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6"
Sep 11 16:47:15.270192 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.270003    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89qzp\" (UniqueName: \"kubernetes.io/projected/c792a287-5a78-4c7b-a6db-a4893b658fc0-kube-api-access-89qzp\") pod \"openshift-lws-operator-fd8ccff4c-cv5g6\" (UID: \"c792a287-5a78-4c7b-a6db-a4893b658fc0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6"
Sep 11 16:47:15.370775 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.370743    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c792a287-5a78-4c7b-a6db-a4893b658fc0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cv5g6\" (UID: \"c792a287-5a78-4c7b-a6db-a4893b658fc0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6"
Sep 11 16:47:15.370942 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.370806    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89qzp\" (UniqueName: \"kubernetes.io/projected/c792a287-5a78-4c7b-a6db-a4893b658fc0-kube-api-access-89qzp\") pod \"openshift-lws-operator-fd8ccff4c-cv5g6\" (UID: \"c792a287-5a78-4c7b-a6db-a4893b658fc0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6"
Sep 11 16:47:15.371101 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.371062    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c792a287-5a78-4c7b-a6db-a4893b658fc0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cv5g6\" (UID: \"c792a287-5a78-4c7b-a6db-a4893b658fc0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6"
Sep 11 16:47:15.384148 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.384127    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89qzp\" (UniqueName: \"kubernetes.io/projected/c792a287-5a78-4c7b-a6db-a4893b658fc0-kube-api-access-89qzp\") pod \"openshift-lws-operator-fd8ccff4c-cv5g6\" (UID: \"c792a287-5a78-4c7b-a6db-a4893b658fc0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6"
Sep 11 16:47:15.496896 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.496806    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6"
Sep 11 16:47:15.772323 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:15.772295    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6"]
Sep 11 16:47:15.775115 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:47:15.775064    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc792a287_5a78_4c7b_a6db_a4893b658fc0.slice/crio-a7af7915ce20a1b8e8d222517e081f7aeac8253985615bdddd4e09ddc7c7f2e5 WatchSource:0}: Error finding container a7af7915ce20a1b8e8d222517e081f7aeac8253985615bdddd4e09ddc7c7f2e5: Status 404 returned error can't find the container with id a7af7915ce20a1b8e8d222517e081f7aeac8253985615bdddd4e09ddc7c7f2e5
Sep 11 16:47:16.286617 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:16.286582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6" event={"ID":"c792a287-5a78-4c7b-a6db-a4893b658fc0","Type":"ContainerStarted","Data":"a7af7915ce20a1b8e8d222517e081f7aeac8253985615bdddd4e09ddc7c7f2e5"}
Sep 11 16:47:19.301708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:19.301674    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6" event={"ID":"c792a287-5a78-4c7b-a6db-a4893b658fc0","Type":"ContainerStarted","Data":"2c3fe1817f1a56ccae8c58986ae07277fe2149a9b8309102d587f519caa3401d"}
Sep 11 16:47:19.326628 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:19.326574    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cv5g6" podStartSLOduration=1.5212437049999998 podStartE2EDuration="4.326553224s" totalImagesPullingTime="2.805309519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:47:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:47:19.324831182 +0000 UTC m=+235.732049829" watchObservedRunningTime="2026-09-11 16:47:19.326553224 +0000 UTC m=+235.733771873"
Sep 11 16:47:20.127860 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:20.127824    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:20.231401 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:20.231346    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:21.925693 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:21.925661    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:36.235909 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:36.235866    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:36.333676 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:36.333641    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:36.838942 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:36.838903    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:37.110051 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.109975    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:37.215047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.215019    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:37.319425 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.319389    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"]
Sep 11 16:47:37.327146 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.327119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.334804 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.334783    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 16:47:37.335925 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.335902    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5x2wg\""
Sep 11 16:47:37.336036 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.335942    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 16:47:37.336244 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.336223    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 16:47:37.337714 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.337696    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 16:47:37.350990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.350950    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"]
Sep 11 16:47:37.352767 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.352728    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8db771b-dba2-4a73-88f9-b2bfce087865-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.352917 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.352781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8db771b-dba2-4a73-88f9-b2bfce087865-tmp\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.352917 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.352861    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qvmn\" (UniqueName: \"kubernetes.io/projected/b8db771b-dba2-4a73-88f9-b2bfce087865-kube-api-access-5qvmn\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.352917 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.352904    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8db771b-dba2-4a73-88f9-b2bfce087865-manifests\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.353086 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.353005    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8db771b-dba2-4a73-88f9-b2bfce087865-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.454293 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.454258    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8db771b-dba2-4a73-88f9-b2bfce087865-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.454293 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.454302    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8db771b-dba2-4a73-88f9-b2bfce087865-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.454550 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.454334    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8db771b-dba2-4a73-88f9-b2bfce087865-tmp\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.454550 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.454365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qvmn\" (UniqueName: \"kubernetes.io/projected/b8db771b-dba2-4a73-88f9-b2bfce087865-kube-api-access-5qvmn\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.454550 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.454412    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8db771b-dba2-4a73-88f9-b2bfce087865-manifests\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.454865 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.454844    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8db771b-dba2-4a73-88f9-b2bfce087865-manifests\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.456891 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.456861    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8db771b-dba2-4a73-88f9-b2bfce087865-tmp\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.457014 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.456981    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8db771b-dba2-4a73-88f9-b2bfce087865-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.457106 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.457016    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8db771b-dba2-4a73-88f9-b2bfce087865-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.467336 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.467307    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qvmn\" (UniqueName: \"kubernetes.io/projected/b8db771b-dba2-4a73-88f9-b2bfce087865-kube-api-access-5qvmn\") pod \"opendatahub-operator-controller-manager-7d67cb4f96-qppv4\" (UID: \"b8db771b-dba2-4a73-88f9-b2bfce087865\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.483132 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.483092    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:37.590176 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.590137    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:37.636549 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.636514    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:37.691684 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.691652    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:37.838930 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.838853    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:37.926617 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.926586    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:37.939658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:37.939623    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"]
Sep 11 16:47:37.953777 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:47:37.953740    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8db771b_dba2_4a73_88f9_b2bfce087865.slice/crio-09a774b9a60760447b415fdc18fa62aafad7d08b40c3236597722f1206ff082e WatchSource:0}: Error finding container 09a774b9a60760447b415fdc18fa62aafad7d08b40c3236597722f1206ff082e: Status 404 returned error can't find the container with id 09a774b9a60760447b415fdc18fa62aafad7d08b40c3236597722f1206ff082e
Sep 11 16:47:38.026251 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:38.026223    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:38.143130 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:38.143103    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:38.282564 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:38.282528    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:38.382868 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:38.382835    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4" event={"ID":"b8db771b-dba2-4a73-88f9-b2bfce087865","Type":"ContainerStarted","Data":"09a774b9a60760447b415fdc18fa62aafad7d08b40c3236597722f1206ff082e"}
Sep 11 16:47:40.766397 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:40.766370    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:40.866233 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:40.866201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:41.398405 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:41.398372    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4" event={"ID":"b8db771b-dba2-4a73-88f9-b2bfce087865","Type":"ContainerStarted","Data":"f837b4aa59dafcbb396fa85498a20fe67ba7970fd239efc632cb4a450bd785cd"}
Sep 11 16:47:42.403770 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:42.403739    2569 generic.go:356] "Generic (PLEG): container finished" podID="b8db771b-dba2-4a73-88f9-b2bfce087865" containerID="f837b4aa59dafcbb396fa85498a20fe67ba7970fd239efc632cb4a450bd785cd" exitCode=0
Sep 11 16:47:42.403770 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:42.403777    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4" event={"ID":"b8db771b-dba2-4a73-88f9-b2bfce087865","Type":"ContainerDied","Data":"f837b4aa59dafcbb396fa85498a20fe67ba7970fd239efc632cb4a450bd785cd"}
Sep 11 16:47:43.409859 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:43.409824    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4" event={"ID":"b8db771b-dba2-4a73-88f9-b2bfce087865","Type":"ContainerStarted","Data":"8c0d2c1584ffc4c99349e0b80774ff1bc40f807e30e668bff28e2b454e1d6bd6"}
Sep 11 16:47:43.410294 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:43.409979    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:43.438545 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:43.438492    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4" podStartSLOduration=2.205148339 podStartE2EDuration="6.438475724s" totalImagesPullingTime="3.185882502s" totalInitContainerRuntime="1.047444883s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:47:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:47:43.437100344 +0000 UTC m=+259.844318995" watchObservedRunningTime="2026-09-11 16:47:43.438475724 +0000 UTC m=+259.845694372"
Sep 11 16:47:44.482592 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:44.482554    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:46.280894 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:46.280797    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:46.381280 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:46.381251    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:54.416780 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.416750    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d67cb4f96-qppv4"
Sep 11 16:47:54.509826 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.509796    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"]
Sep 11 16:47:54.516839 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.516819    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.519241 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.519213    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 16:47:54.519241 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.519219    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 16:47:54.519403 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.519324    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 16:47:54.519563 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.519551    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-nv6xb\""
Sep 11 16:47:54.525609 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.525587    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"]
Sep 11 16:47:54.594178 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.594143    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-metrics-cert\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.594359 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.594184    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x62cp\" (UniqueName: \"kubernetes.io/projected/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-kube-api-access-x62cp\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.594359 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.594258    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-cert\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.594440 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.594376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-manager-config\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.694993 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.694909    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-metrics-cert\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.694993 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.694946    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x62cp\" (UniqueName: \"kubernetes.io/projected/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-kube-api-access-x62cp\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.695240 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.695054    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-cert\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.695240 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.695099    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-manager-config\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.695674 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.695649    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-manager-config\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.697418 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.697396    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-cert\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.697506 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.697433    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-metrics-cert\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.704178 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.704157    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x62cp\" (UniqueName: \"kubernetes.io/projected/648e3cb0-db9c-42db-b080-9f4b0d4f4d5f-kube-api-access-x62cp\") pod \"lws-controller-manager-67c876df9b-ghrtp\" (UID: \"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f\") " pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:54.827035 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:54.826992    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:55.100259 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:55.100180    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"]
Sep 11 16:47:55.103644 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:47:55.103617    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648e3cb0_db9c_42db_b080_9f4b0d4f4d5f.slice/crio-cfe904e63ef8ca5f6c5978006ff681c09b18e1504c15ae77f5a67c578638a800 WatchSource:0}: Error finding container cfe904e63ef8ca5f6c5978006ff681c09b18e1504c15ae77f5a67c578638a800: Status 404 returned error can't find the container with id cfe904e63ef8ca5f6c5978006ff681c09b18e1504c15ae77f5a67c578638a800
Sep 11 16:47:55.464539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:55.464504    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp" event={"ID":"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f","Type":"ContainerStarted","Data":"cfe904e63ef8ca5f6c5978006ff681c09b18e1504c15ae77f5a67c578638a800"}
Sep 11 16:47:56.171205 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:56.171164    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:56.300217 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:56.300178    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:56.601285 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:56.601185    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:56.974449 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:56.974405    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:57.476972 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:57.476934    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp" event={"ID":"648e3cb0-db9c-42db-b080-9f4b0d4f4d5f","Type":"ContainerStarted","Data":"3490b13dc857d588d01db5e9aafe562833976e84c6c8eaf765f7aea98895ec1d"}
Sep 11 16:47:57.477166 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:57.477028    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:47:57.504224 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:57.504169    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp" podStartSLOduration=1.223352068 podStartE2EDuration="3.50415068s" totalImagesPullingTime="2.280798612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:47:57.501590203 +0000 UTC m=+273.908808869" watchObservedRunningTime="2026-09-11 16:47:57.50415068 +0000 UTC m=+273.911369327"
Sep 11 16:47:58.241137 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:58.241085    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:58.337230 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:58.337203    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:47:59.912763 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:47:59.912732    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:03.208135 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:03.208067    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:03.314558 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:03.314509    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:04.198389 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.198359    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"]
Sep 11 16:48:04.201885 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.201866    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.207228 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.207203    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 16:48:04.207372 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.207276    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 16:48:04.207372 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.207291    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 16:48:04.207642 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.207623    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-69thq\""
Sep 11 16:48:04.220441 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.220415    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"]
Sep 11 16:48:04.232446 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.232418    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:04.284932 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.284888    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.284932 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.284941    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.285217 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.284965    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.285217 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.285019    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.285217 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.285136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.285217 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.285160    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7249\" (UniqueName: \"kubernetes.io/projected/8a4fa0d6-9426-42f7-82b9-18ef74086496-kube-api-access-w7249\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.285217 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.285178    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.285217 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.285200    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4fa0d6-9426-42f7-82b9-18ef74086496-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.285417 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.285226    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.386398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386352    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.386398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386401    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.386642 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386422    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.386642 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386437    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.386642 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386459    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.386642 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386485    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7249\" (UniqueName: \"kubernetes.io/projected/8a4fa0d6-9426-42f7-82b9-18ef74086496-kube-api-access-w7249\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.386642 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386608    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.386899 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386661    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4fa0d6-9426-42f7-82b9-18ef74086496-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.386899 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386690    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.386899 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386833    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.386899 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386884    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.387052 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.386955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.387052 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.387039    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.387381 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.387363    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4fa0d6-9426-42f7-82b9-18ef74086496-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.389340 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.389310    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.389469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.389452    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.405843 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.405813    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4fa0d6-9426-42f7-82b9-18ef74086496-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.406169 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.406146    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7249\" (UniqueName: \"kubernetes.io/projected/8a4fa0d6-9426-42f7-82b9-18ef74086496-kube-api-access-w7249\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4\" (UID: \"8a4fa0d6-9426-42f7-82b9-18ef74086496\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.483926 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.483847    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:04.512172 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.512141    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:04.597911 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.597858    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:04.794110 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:04.794006    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"]
Sep 11 16:48:04.797265 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:48:04.797236    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a4fa0d6_9426_42f7_82b9_18ef74086496.slice/crio-f07c1980d22a537821e72cca0c99d6dcae718a6f2799e0c87321dcfbf088efa9 WatchSource:0}: Error finding container f07c1980d22a537821e72cca0c99d6dcae718a6f2799e0c87321dcfbf088efa9: Status 404 returned error can't find the container with id f07c1980d22a537821e72cca0c99d6dcae718a6f2799e0c87321dcfbf088efa9
Sep 11 16:48:05.515061 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:05.514981    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4" event={"ID":"8a4fa0d6-9426-42f7-82b9-18ef74086496","Type":"ContainerStarted","Data":"f07c1980d22a537821e72cca0c99d6dcae718a6f2799e0c87321dcfbf088efa9"}
Sep 11 16:48:07.277660 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.277622    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-96pzb"]
Sep 11 16:48:07.288881 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.288685    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-96pzb"
Sep 11 16:48:07.291237 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.291203    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-96pzb"]
Sep 11 16:48:07.291644 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.291536    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-kfp2b\""
Sep 11 16:48:07.293454 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.293432    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 16:48:07.340472 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.340412    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:07.409561 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.409528    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:48:07.409647 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.409599    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:48:07.409647 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.409629    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:48:07.416725 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.416697    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20d779ce-f46a-4938-85e7-bbab7575ebf6-cert\") pod \"odh-model-controller-7998c67dc4-96pzb\" (UID: \"20d779ce-f46a-4938-85e7-bbab7575ebf6\") " pod="opendatahub/odh-model-controller-7998c67dc4-96pzb"
Sep 11 16:48:07.416820 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.416802    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqmlq\" (UniqueName: \"kubernetes.io/projected/20d779ce-f46a-4938-85e7-bbab7575ebf6-kube-api-access-tqmlq\") pod \"odh-model-controller-7998c67dc4-96pzb\" (UID: \"20d779ce-f46a-4938-85e7-bbab7575ebf6\") " pod="opendatahub/odh-model-controller-7998c67dc4-96pzb"
Sep 11 16:48:07.517436 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.517398    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqmlq\" (UniqueName: \"kubernetes.io/projected/20d779ce-f46a-4938-85e7-bbab7575ebf6-kube-api-access-tqmlq\") pod \"odh-model-controller-7998c67dc4-96pzb\" (UID: \"20d779ce-f46a-4938-85e7-bbab7575ebf6\") " pod="opendatahub/odh-model-controller-7998c67dc4-96pzb"
Sep 11 16:48:07.517622 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.517501    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20d779ce-f46a-4938-85e7-bbab7575ebf6-cert\") pod \"odh-model-controller-7998c67dc4-96pzb\" (UID: \"20d779ce-f46a-4938-85e7-bbab7575ebf6\") " pod="opendatahub/odh-model-controller-7998c67dc4-96pzb"
Sep 11 16:48:07.517688 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:48:07.517658    2569 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 11 16:48:07.517780 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:48:07.517767    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20d779ce-f46a-4938-85e7-bbab7575ebf6-cert podName:20d779ce-f46a-4938-85e7-bbab7575ebf6 nodeName:}" failed. No retries permitted until 2026-09-11 16:48:08.017733816 +0000 UTC m=+284.424952455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/20d779ce-f46a-4938-85e7-bbab7575ebf6-cert") pod "odh-model-controller-7998c67dc4-96pzb" (UID: "20d779ce-f46a-4938-85e7-bbab7575ebf6") : secret "odh-model-controller-webhook-cert" not found
Sep 11 16:48:07.527641 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.527602    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4" event={"ID":"8a4fa0d6-9426-42f7-82b9-18ef74086496","Type":"ContainerStarted","Data":"8caf6aeba8e391cbc6902e1471983ca726e1bff1f5610e69f5381021f348e784"}
Sep 11 16:48:07.529254 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.529190    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqmlq\" (UniqueName: \"kubernetes.io/projected/20d779ce-f46a-4938-85e7-bbab7575ebf6-kube-api-access-tqmlq\") pod \"odh-model-controller-7998c67dc4-96pzb\" (UID: \"20d779ce-f46a-4938-85e7-bbab7575ebf6\") " pod="opendatahub/odh-model-controller-7998c67dc4-96pzb"
Sep 11 16:48:07.551544 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.551473    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4" podStartSLOduration=0.941261873 podStartE2EDuration="3.551450749s" totalImagesPullingTime="2.610188876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:48:07.550244226 +0000 UTC m=+283.957462893" watchObservedRunningTime="2026-09-11 16:48:07.551450749 +0000 UTC m=+283.958669397"
Sep 11 16:48:07.939383 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.939345    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r548q"]
Sep 11 16:48:07.942685 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.942665    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-r548q"
Sep 11 16:48:07.944888 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.944859    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 16:48:07.955614 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:07.955584    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r548q"]
Sep 11 16:48:08.022241 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.022205    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45022cad-a452-41a1-a0cd-ed799faa9fac-tls-certs\") pod \"model-serving-api-54d89bd5fc-r548q\" (UID: \"45022cad-a452-41a1-a0cd-ed799faa9fac\") " pod="opendatahub/model-serving-api-54d89bd5fc-r548q"
Sep 11 16:48:08.022421 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.022259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20d779ce-f46a-4938-85e7-bbab7575ebf6-cert\") pod \"odh-model-controller-7998c67dc4-96pzb\" (UID: \"20d779ce-f46a-4938-85e7-bbab7575ebf6\") " pod="opendatahub/odh-model-controller-7998c67dc4-96pzb"
Sep 11 16:48:08.022421 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.022380    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxr9r\" (UniqueName: \"kubernetes.io/projected/45022cad-a452-41a1-a0cd-ed799faa9fac-kube-api-access-nxr9r\") pod \"model-serving-api-54d89bd5fc-r548q\" (UID: \"45022cad-a452-41a1-a0cd-ed799faa9fac\") " pod="opendatahub/model-serving-api-54d89bd5fc-r548q"
Sep 11 16:48:08.024718 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.024698    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20d779ce-f46a-4938-85e7-bbab7575ebf6-cert\") pod \"odh-model-controller-7998c67dc4-96pzb\" (UID: \"20d779ce-f46a-4938-85e7-bbab7575ebf6\") " pod="opendatahub/odh-model-controller-7998c67dc4-96pzb"
Sep 11 16:48:08.123411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.123378    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxr9r\" (UniqueName: \"kubernetes.io/projected/45022cad-a452-41a1-a0cd-ed799faa9fac-kube-api-access-nxr9r\") pod \"model-serving-api-54d89bd5fc-r548q\" (UID: \"45022cad-a452-41a1-a0cd-ed799faa9fac\") " pod="opendatahub/model-serving-api-54d89bd5fc-r548q"
Sep 11 16:48:08.123587 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.123426    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45022cad-a452-41a1-a0cd-ed799faa9fac-tls-certs\") pod \"model-serving-api-54d89bd5fc-r548q\" (UID: \"45022cad-a452-41a1-a0cd-ed799faa9fac\") " pod="opendatahub/model-serving-api-54d89bd5fc-r548q"
Sep 11 16:48:08.125918 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.125899    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45022cad-a452-41a1-a0cd-ed799faa9fac-tls-certs\") pod \"model-serving-api-54d89bd5fc-r548q\" (UID: \"45022cad-a452-41a1-a0cd-ed799faa9fac\") " pod="opendatahub/model-serving-api-54d89bd5fc-r548q"
Sep 11 16:48:08.133479 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.133452    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxr9r\" (UniqueName: \"kubernetes.io/projected/45022cad-a452-41a1-a0cd-ed799faa9fac-kube-api-access-nxr9r\") pod \"model-serving-api-54d89bd5fc-r548q\" (UID: \"45022cad-a452-41a1-a0cd-ed799faa9fac\") " pod="opendatahub/model-serving-api-54d89bd5fc-r548q"
Sep 11 16:48:08.203563 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.203465    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-96pzb"
Sep 11 16:48:08.253925 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.253892    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-r548q"
Sep 11 16:48:08.485166 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.485058    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-67c876df9b-ghrtp"
Sep 11 16:48:08.485956 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.485937    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-96pzb"]
Sep 11 16:48:08.488095 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:48:08.488059    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d779ce_f46a_4938_85e7_bbab7575ebf6.slice/crio-95acd8bf33b5d8715d87b8b1c41df8fb7db2120137ad2f6aa624178b72ec34b9 WatchSource:0}: Error finding container 95acd8bf33b5d8715d87b8b1c41df8fb7db2120137ad2f6aa624178b72ec34b9: Status 404 returned error can't find the container with id 95acd8bf33b5d8715d87b8b1c41df8fb7db2120137ad2f6aa624178b72ec34b9
Sep 11 16:48:08.513156 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.513112    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:08.518975 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.518949    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:08.534967 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.534929    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-96pzb" event={"ID":"20d779ce-f46a-4938-85e7-bbab7575ebf6","Type":"ContainerStarted","Data":"95acd8bf33b5d8715d87b8b1c41df8fb7db2120137ad2f6aa624178b72ec34b9"}
Sep 11 16:48:08.535251 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.535232    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:08.536449 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.536427    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4"
Sep 11 16:48:08.546309 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.546273    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r548q"]
Sep 11 16:48:08.550478 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:48:08.550433    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45022cad_a452_41a1_a0cd_ed799faa9fac.slice/crio-ea5db7a3c472830e7972f44678bc8757c6448e1edf4b92be20493cba55d87630 WatchSource:0}: Error finding container ea5db7a3c472830e7972f44678bc8757c6448e1edf4b92be20493cba55d87630: Status 404 returned error can't find the container with id ea5db7a3c472830e7972f44678bc8757c6448e1edf4b92be20493cba55d87630
Sep 11 16:48:08.656019 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:08.655984    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:09.542969 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:09.542929    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-r548q" event={"ID":"45022cad-a452-41a1-a0cd-ed799faa9fac","Type":"ContainerStarted","Data":"ea5db7a3c472830e7972f44678bc8757c6448e1edf4b92be20493cba55d87630"}
Sep 11 16:48:12.560364 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:12.560277    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-96pzb" event={"ID":"20d779ce-f46a-4938-85e7-bbab7575ebf6","Type":"ContainerStarted","Data":"e3b8cbb10a8d4f04224b89750d7792482a2e0dc4c402235f7939a3786628c944"}
Sep 11 16:48:12.560774 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:12.560395    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-96pzb"
Sep 11 16:48:12.561894 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:12.561872    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-r548q" event={"ID":"45022cad-a452-41a1-a0cd-ed799faa9fac","Type":"ContainerStarted","Data":"53c820b3bebdc5f374f111fe7e56f4acc500a8a8ec6f54e54454701ffd587c6e"}
Sep 11 16:48:12.562017 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:12.561919    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-r548q"
Sep 11 16:48:12.601868 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:12.601819    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-96pzb" podStartSLOduration=1.908874652 podStartE2EDuration="5.601803897s" totalImagesPullingTime="3.692929245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:48:12.599337755 +0000 UTC m=+289.006556421" watchObservedRunningTime="2026-09-11 16:48:12.601803897 +0000 UTC m=+289.009022587"
Sep 11 16:48:12.627810 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:12.627765    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-r548q" podStartSLOduration=1.958559863 podStartE2EDuration="5.627749539s" totalImagesPullingTime="3.669189676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:48:12.62630436 +0000 UTC m=+289.033523009" watchObservedRunningTime="2026-09-11 16:48:12.627749539 +0000 UTC m=+289.034968187"
Sep 11 16:48:18.500416 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.500385    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:18.642448 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.642412    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"]
Sep 11 16:48:18.646033 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.646014    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"
Sep 11 16:48:18.647477 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.647446    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:18.652786 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.652660    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 11 16:48:18.652786 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.652704    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wgpz4\""
Sep 11 16:48:18.683699 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.683664    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"]
Sep 11 16:48:18.724473 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.724430    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5-cert\") pod \"kserve-controller-manager-d6f4b65b8-s58q5\" (UID: \"0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"
Sep 11 16:48:18.724473 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.724467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6scnr\" (UniqueName: \"kubernetes.io/projected/0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5-kube-api-access-6scnr\") pod \"kserve-controller-manager-d6f4b65b8-s58q5\" (UID: \"0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"
Sep 11 16:48:18.798015 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.797891    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:18.825006 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.824977    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5-cert\") pod \"kserve-controller-manager-d6f4b65b8-s58q5\" (UID: \"0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"
Sep 11 16:48:18.825163 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.825008    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6scnr\" (UniqueName: \"kubernetes.io/projected/0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5-kube-api-access-6scnr\") pod \"kserve-controller-manager-d6f4b65b8-s58q5\" (UID: \"0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"
Sep 11 16:48:18.827466 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.827439    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5-cert\") pod \"kserve-controller-manager-d6f4b65b8-s58q5\" (UID: \"0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"
Sep 11 16:48:18.865008 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.864977    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6scnr\" (UniqueName: \"kubernetes.io/projected/0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5-kube-api-access-6scnr\") pod \"kserve-controller-manager-d6f4b65b8-s58q5\" (UID: \"0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"
Sep 11 16:48:18.956303 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.956274    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"
Sep 11 16:48:18.978483 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:18.978438    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:19.086587 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:19.086508    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:19.289644 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:19.289548    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"]
Sep 11 16:48:19.292494 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:48:19.292458    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b4dd94c_94bd_4fb6_83ef_ec0f2cb725e5.slice/crio-b49f7d61cd7afb10303f5ec87e19926a00874775840933feadb03ba3114d683f WatchSource:0}: Error finding container b49f7d61cd7afb10303f5ec87e19926a00874775840933feadb03ba3114d683f: Status 404 returned error can't find the container with id b49f7d61cd7afb10303f5ec87e19926a00874775840933feadb03ba3114d683f
Sep 11 16:48:19.592973 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:19.592937    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5" event={"ID":"0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5","Type":"ContainerStarted","Data":"b49f7d61cd7afb10303f5ec87e19926a00874775840933feadb03ba3114d683f"}
Sep 11 16:48:19.985861 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:19.985829    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:20.081551 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:20.081514    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:20.189284 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:20.189254    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:48:22.610977 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:22.610933    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5" event={"ID":"0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5","Type":"ContainerStarted","Data":"fcba805297e1f49d09a3c9f202e1001145612dde793a9982fb498d4fc344c136"}
Sep 11 16:48:22.611483 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:22.611121    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"
Sep 11 16:48:22.644417 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:22.644366    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5" podStartSLOduration=2.071968126 podStartE2EDuration="4.644346284s" totalImagesPullingTime="2.572378158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:48:22.643684609 +0000 UTC m=+299.050903282" watchObservedRunningTime="2026-09-11 16:48:22.644346284 +0000 UTC m=+299.051564934"
Sep 11 16:48:23.568062 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:23.568033    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-96pzb"
Sep 11 16:48:23.571281 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:23.571255    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-r548q"
Sep 11 16:48:23.970533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:23.970501    2569 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 16:48:53.621000 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:48:53.620962    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s58q5"
Sep 11 16:49:08.240700 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:08.240668    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:08.320008 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:08.319971    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:10.224831 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:10.224798    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:10.287225 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:10.287199    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:11.966007 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:11.965963    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:12.060562 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:12.060523    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:22.403426 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:22.403355    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:24.495698 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:24.495667    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"]
Sep 11 16:49:24.498900 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:24.498885    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:24.501399 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:24.501378    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 16:49:24.501621 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:24.501605    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-wh2zv\""
Sep 11 16:49:24.502048 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:24.502031    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 16:49:24.516294 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:24.516272    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"]
Sep 11 16:49:24.588519 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:24.588486    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sclm2\" (UniqueName: \"kubernetes.io/projected/a00baf4d-df4b-4a57-bce3-34fa7ab6f2da-kube-api-access-sclm2\") pod \"limitador-operator-controller-manager-74495c69f8-ndnsc\" (UID: \"a00baf4d-df4b-4a57-bce3-34fa7ab6f2da\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:24.688979 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:24.688942    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sclm2\" (UniqueName: \"kubernetes.io/projected/a00baf4d-df4b-4a57-bce3-34fa7ab6f2da-kube-api-access-sclm2\") pod \"limitador-operator-controller-manager-74495c69f8-ndnsc\" (UID: \"a00baf4d-df4b-4a57-bce3-34fa7ab6f2da\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:24.698970 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:24.698938    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sclm2\" (UniqueName: \"kubernetes.io/projected/a00baf4d-df4b-4a57-bce3-34fa7ab6f2da-kube-api-access-sclm2\") pod \"limitador-operator-controller-manager-74495c69f8-ndnsc\" (UID: \"a00baf4d-df4b-4a57-bce3-34fa7ab6f2da\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:24.806431 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:24.806347    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:24.809340 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:24.809322    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:25.085835 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:25.085807    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"]
Sep 11 16:49:25.087255 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:49:25.087227    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00baf4d_df4b_4a57_bce3_34fa7ab6f2da.slice/crio-cbdaad7fa5f8801e5cb9e1df01b1590b4fdcfac9411f7e56b09981a4363e85e3 WatchSource:0}: Error finding container cbdaad7fa5f8801e5cb9e1df01b1590b4fdcfac9411f7e56b09981a4363e85e3: Status 404 returned error can't find the container with id cbdaad7fa5f8801e5cb9e1df01b1590b4fdcfac9411f7e56b09981a4363e85e3
Sep 11 16:49:25.089623 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:25.089604    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 16:49:25.298315 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:25.298285    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:25.893506 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:25.893471    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc" event={"ID":"a00baf4d-df4b-4a57-bce3-34fa7ab6f2da","Type":"ContainerStarted","Data":"cbdaad7fa5f8801e5cb9e1df01b1590b4fdcfac9411f7e56b09981a4363e85e3"}
Sep 11 16:49:26.655331 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:26.655277    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:26.756166 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:26.756130    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:27.047555 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:27.047479    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:27.906184 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:27.906146    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc" event={"ID":"a00baf4d-df4b-4a57-bce3-34fa7ab6f2da","Type":"ContainerStarted","Data":"a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0"}
Sep 11 16:49:27.906408 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:27.906388    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:27.931271 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:27.931217    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc" podStartSLOduration=1.268575931 podStartE2EDuration="3.931197845s" totalImagesPullingTime="2.662621914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:27.929004969 +0000 UTC m=+364.336223636" watchObservedRunningTime="2026-09-11 16:49:27.931197845 +0000 UTC m=+364.338416495"
Sep 11 16:49:28.224621 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:28.224568    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:28.308637 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:28.308604    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:29.489721 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:29.489690    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:30.413240 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:30.413196    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:30.515693 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:30.515653    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:30.822286 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:30.822192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:30.913902 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:30.913861    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:31.215301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:31.215270    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:31.332250 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:31.332221    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:36.424011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:36.423978    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:36.524141 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:36.524101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:38.071398 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:38.071358    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:38.168568 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:38.168521    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:38.269145 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:38.269098    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:38.913496 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:38.913465    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:40.236711 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:40.236679    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:40.346543 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:40.346511    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:40.732866 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:40.732822    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:40.842821 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:40.842782    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:42.165727 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:42.165683    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:42.264209 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:42.264158    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:43.494064 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:43.494031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:43.592908 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:43.592879    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:49.750728 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:49.750692    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:49.851377 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:49.851348    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:49.952362 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:49.952333    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:52.168245 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.168200    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"]
Sep 11 16:49:52.175335 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.175315    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:52.181980 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.178834    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ngvwn\""
Sep 11 16:49:52.207547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.207520    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"]
Sep 11 16:49:52.331767 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.331736    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5228372-7634-4439-ba4e-bc3fdf7463de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-lbkvl\" (UID: \"b5228372-7634-4439-ba4e-bc3fdf7463de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:52.331931 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.331800    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp7r7\" (UniqueName: \"kubernetes.io/projected/b5228372-7634-4439-ba4e-bc3fdf7463de-kube-api-access-hp7r7\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-lbkvl\" (UID: \"b5228372-7634-4439-ba4e-bc3fdf7463de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:52.433381 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.433296    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp7r7\" (UniqueName: \"kubernetes.io/projected/b5228372-7634-4439-ba4e-bc3fdf7463de-kube-api-access-hp7r7\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-lbkvl\" (UID: \"b5228372-7634-4439-ba4e-bc3fdf7463de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:52.433381 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.433364    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5228372-7634-4439-ba4e-bc3fdf7463de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-lbkvl\" (UID: \"b5228372-7634-4439-ba4e-bc3fdf7463de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:52.433745 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.433725    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5228372-7634-4439-ba4e-bc3fdf7463de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-lbkvl\" (UID: \"b5228372-7634-4439-ba4e-bc3fdf7463de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:52.447816 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.447793    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp7r7\" (UniqueName: \"kubernetes.io/projected/b5228372-7634-4439-ba4e-bc3fdf7463de-kube-api-access-hp7r7\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-lbkvl\" (UID: \"b5228372-7634-4439-ba4e-bc3fdf7463de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:52.490747 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.490719    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:52.769966 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:52.769930    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"]
Sep 11 16:49:52.772889 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:49:52.772853    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5228372_7634_4439_ba4e_bc3fdf7463de.slice/crio-2e506fb119e1ae4dd4ef25189e4136bf61666790bf37bbfde0e2e17f20b3d9c2 WatchSource:0}: Error finding container 2e506fb119e1ae4dd4ef25189e4136bf61666790bf37bbfde0e2e17f20b3d9c2: Status 404 returned error can't find the container with id 2e506fb119e1ae4dd4ef25189e4136bf61666790bf37bbfde0e2e17f20b3d9c2
Sep 11 16:49:53.026343 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.026263    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl" event={"ID":"b5228372-7634-4439-ba4e-bc3fdf7463de","Type":"ContainerStarted","Data":"2e506fb119e1ae4dd4ef25189e4136bf61666790bf37bbfde0e2e17f20b3d9c2"}
Sep 11 16:49:53.406178 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.406147    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"]
Sep 11 16:49:53.414325 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.414297    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"]
Sep 11 16:49:53.450452 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.450424    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"]
Sep 11 16:49:53.455796 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.455777    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:49:53.461933 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.461910    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"]
Sep 11 16:49:53.462121 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.462100    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc" podUID="a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" containerName="manager" containerID="cri-o://a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0" gracePeriod=2
Sep 11 16:49:53.475845 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.475821    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"]
Sep 11 16:49:53.482516 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.482490    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"]
Sep 11 16:49:53.494913 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.494886    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9"]
Sep 11 16:49:53.495284 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.495266    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" containerName="manager"
Sep 11 16:49:53.495284 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.495284    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" containerName="manager"
Sep 11 16:49:53.495445 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.495372    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" containerName="manager"
Sep 11 16:49:53.498818 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.498801    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9"
Sep 11 16:49:53.514867 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.514837    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9"]
Sep 11 16:49:53.515670 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:49:53.515638    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ndnsc\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:53.643785 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.643685    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00334d37-7c5b-4927-8ada-623ed2f4136a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-5mv42\" (UID: \"00334d37-7c5b-4927-8ada-623ed2f4136a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:49:53.643785 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.643737    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7j6j\" (UniqueName: \"kubernetes.io/projected/4ea846a3-9406-4e8b-a1e4-d178be88b5d5-kube-api-access-g7j6j\") pod \"limitador-operator-controller-manager-74495c69f8-2rmt9\" (UID: \"4ea846a3-9406-4e8b-a1e4-d178be88b5d5\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9"
Sep 11 16:49:53.643982 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.643946    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v4hs\" (UniqueName: \"kubernetes.io/projected/00334d37-7c5b-4927-8ada-623ed2f4136a-kube-api-access-6v4hs\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-5mv42\" (UID: \"00334d37-7c5b-4927-8ada-623ed2f4136a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:49:53.706554 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.706522    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:53.709036 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:49:53.709000    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ndnsc\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:53.744500 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.744466    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v4hs\" (UniqueName: \"kubernetes.io/projected/00334d37-7c5b-4927-8ada-623ed2f4136a-kube-api-access-6v4hs\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-5mv42\" (UID: \"00334d37-7c5b-4927-8ada-623ed2f4136a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:49:53.744672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.744524    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00334d37-7c5b-4927-8ada-623ed2f4136a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-5mv42\" (UID: \"00334d37-7c5b-4927-8ada-623ed2f4136a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:49:53.744672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.744549    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7j6j\" (UniqueName: \"kubernetes.io/projected/4ea846a3-9406-4e8b-a1e4-d178be88b5d5-kube-api-access-g7j6j\") pod \"limitador-operator-controller-manager-74495c69f8-2rmt9\" (UID: \"4ea846a3-9406-4e8b-a1e4-d178be88b5d5\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9"
Sep 11 16:49:53.744977 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.744955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00334d37-7c5b-4927-8ada-623ed2f4136a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-5mv42\" (UID: \"00334d37-7c5b-4927-8ada-623ed2f4136a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:49:53.762703 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.762659    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7j6j\" (UniqueName: \"kubernetes.io/projected/4ea846a3-9406-4e8b-a1e4-d178be88b5d5-kube-api-access-g7j6j\") pod \"limitador-operator-controller-manager-74495c69f8-2rmt9\" (UID: \"4ea846a3-9406-4e8b-a1e4-d178be88b5d5\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9"
Sep 11 16:49:53.763041 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.763007    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v4hs\" (UniqueName: \"kubernetes.io/projected/00334d37-7c5b-4927-8ada-623ed2f4136a-kube-api-access-6v4hs\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-5mv42\" (UID: \"00334d37-7c5b-4927-8ada-623ed2f4136a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:49:53.766800 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.766777    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:49:53.834835 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.834800    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9"
Sep 11 16:49:53.845622 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.845596    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sclm2\" (UniqueName: \"kubernetes.io/projected/a00baf4d-df4b-4a57-bce3-34fa7ab6f2da-kube-api-access-sclm2\") pod \"a00baf4d-df4b-4a57-bce3-34fa7ab6f2da\" (UID: \"a00baf4d-df4b-4a57-bce3-34fa7ab6f2da\") "
Sep 11 16:49:53.848827 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.848788    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a00baf4d-df4b-4a57-bce3-34fa7ab6f2da-kube-api-access-sclm2" (OuterVolumeSpecName: "kube-api-access-sclm2") pod "a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" (UID: "a00baf4d-df4b-4a57-bce3-34fa7ab6f2da"). InnerVolumeSpecName "kube-api-access-sclm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:49:53.946927 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:53.946842    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sclm2\" (UniqueName: \"kubernetes.io/projected/a00baf4d-df4b-4a57-bce3-34fa7ab6f2da-kube-api-access-sclm2\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:49:54.034290 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:54.034223    2569 generic.go:356] "Generic (PLEG): container finished" podID="a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" containerID="a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0" exitCode=0
Sep 11 16:49:54.034290 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:54.034292    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:54.034505 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:54.034306    2569 scope.go:117] "RemoveContainer" containerID="a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0"
Sep 11 16:49:54.036742 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:49:54.036715    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ndnsc\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:54.044877 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:54.044853    2569 scope.go:117] "RemoveContainer" containerID="a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0"
Sep 11 16:49:54.045209 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:49:54.045186    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0\": container with ID starting with a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0 not found: ID does not exist" containerID="a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0"
Sep 11 16:49:54.045291 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:54.045219    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0"} err="failed to get container status \"a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0\": rpc error: code = NotFound desc = could not find container \"a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0\": container with ID starting with a1ca441a80cd0e6dd8c74f119e3bb8a183318c8e6ee6761911513bb1a9fcddf0 not found: ID does not exist"
Sep 11 16:49:54.045814 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:49:54.045793    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ndnsc\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:54.052511 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:54.052486    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"]
Sep 11 16:49:54.055130 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:49:54.055103    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00334d37_7c5b_4927_8ada_623ed2f4136a.slice/crio-18c41a1d00057a2759b31cda8f6a327bc5e3521245ddd8d524e16cf983394ba9 WatchSource:0}: Error finding container 18c41a1d00057a2759b31cda8f6a327bc5e3521245ddd8d524e16cf983394ba9: Status 404 returned error can't find the container with id 18c41a1d00057a2759b31cda8f6a327bc5e3521245ddd8d524e16cf983394ba9
Sep 11 16:49:54.124877 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:54.124845    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9"]
Sep 11 16:49:54.128748 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:49:54.128717    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea846a3_9406_4e8b_a1e4_d178be88b5d5.slice/crio-e9398a5ee0b9d7982981b568a2c13362f54f145f8411a3aa55296d478e137e6a WatchSource:0}: Error finding container e9398a5ee0b9d7982981b568a2c13362f54f145f8411a3aa55296d478e137e6a: Status 404 returned error can't find the container with id e9398a5ee0b9d7982981b568a2c13362f54f145f8411a3aa55296d478e137e6a
Sep 11 16:49:54.204878 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:49:54.204847    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ndnsc\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ndnsc"
Sep 11 16:49:54.207593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:54.207563    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a00baf4d-df4b-4a57-bce3-34fa7ab6f2da" path="/var/lib/kubelet/pods/a00baf4d-df4b-4a57-bce3-34fa7ab6f2da/volumes"
Sep 11 16:49:55.043068 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:55.042965    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42" event={"ID":"00334d37-7c5b-4927-8ada-623ed2f4136a","Type":"ContainerStarted","Data":"18c41a1d00057a2759b31cda8f6a327bc5e3521245ddd8d524e16cf983394ba9"}
Sep 11 16:49:55.045122 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:55.045062    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9" event={"ID":"4ea846a3-9406-4e8b-a1e4-d178be88b5d5","Type":"ContainerStarted","Data":"25e28ce32975ffdf603c0ca4ee5880246f3eb4d753f65b863fc6574ee38ab23d"}
Sep 11 16:49:55.045241 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:55.045144    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9" event={"ID":"4ea846a3-9406-4e8b-a1e4-d178be88b5d5","Type":"ContainerStarted","Data":"e9398a5ee0b9d7982981b568a2c13362f54f145f8411a3aa55296d478e137e6a"}
Sep 11 16:49:55.045241 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:55.045203    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9"
Sep 11 16:49:55.078009 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:55.077963    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9" podStartSLOduration=2.077945093 podStartE2EDuration="2.077945093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:55.075659318 +0000 UTC m=+391.482878002" watchObservedRunningTime="2026-09-11 16:49:55.077945093 +0000 UTC m=+391.485163740"
Sep 11 16:49:56.473887 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:56.473818    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:56.581422 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:56.581369    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:49:57.060308 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:57.060280    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl" podUID="b5228372-7634-4439-ba4e-bc3fdf7463de" containerName="manager" containerID="cri-o://3901a40c76812c845e0da0f7b1e8d8a48b0bdf906ccaa4bcdac3150abcbf674e" gracePeriod=2
Sep 11 16:49:57.307939 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:57.307909    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:57.310145 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:49:57.310118    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-lbkvl\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="b5228372-7634-4439-ba4e-bc3fdf7463de" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:57.482500 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:57.482397    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp7r7\" (UniqueName: \"kubernetes.io/projected/b5228372-7634-4439-ba4e-bc3fdf7463de-kube-api-access-hp7r7\") pod \"b5228372-7634-4439-ba4e-bc3fdf7463de\" (UID: \"b5228372-7634-4439-ba4e-bc3fdf7463de\") "
Sep 11 16:49:57.482500 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:57.482460    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5228372-7634-4439-ba4e-bc3fdf7463de-extensions-socket-volume\") pod \"b5228372-7634-4439-ba4e-bc3fdf7463de\" (UID: \"b5228372-7634-4439-ba4e-bc3fdf7463de\") "
Sep 11 16:49:57.482976 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:57.482792    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5228372-7634-4439-ba4e-bc3fdf7463de-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b5228372-7634-4439-ba4e-bc3fdf7463de" (UID: "b5228372-7634-4439-ba4e-bc3fdf7463de"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:49:57.484504 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:57.484482    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5228372-7634-4439-ba4e-bc3fdf7463de-kube-api-access-hp7r7" (OuterVolumeSpecName: "kube-api-access-hp7r7") pod "b5228372-7634-4439-ba4e-bc3fdf7463de" (UID: "b5228372-7634-4439-ba4e-bc3fdf7463de"). InnerVolumeSpecName "kube-api-access-hp7r7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:49:57.583407 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:57.583362    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp7r7\" (UniqueName: \"kubernetes.io/projected/b5228372-7634-4439-ba4e-bc3fdf7463de-kube-api-access-hp7r7\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:49:57.583407 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:57.583398    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5228372-7634-4439-ba4e-bc3fdf7463de-extensions-socket-volume\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:49:58.066246 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:58.066207    2569 generic.go:356] "Generic (PLEG): container finished" podID="b5228372-7634-4439-ba4e-bc3fdf7463de" containerID="3901a40c76812c845e0da0f7b1e8d8a48b0bdf906ccaa4bcdac3150abcbf674e" exitCode=2
Sep 11 16:49:58.066415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:58.066256    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:58.066415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:58.066310    2569 scope.go:117] "RemoveContainer" containerID="3901a40c76812c845e0da0f7b1e8d8a48b0bdf906ccaa4bcdac3150abcbf674e"
Sep 11 16:49:58.068325 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:58.068299    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42" event={"ID":"00334d37-7c5b-4927-8ada-623ed2f4136a","Type":"ContainerStarted","Data":"7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf"}
Sep 11 16:49:58.068465 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:49:58.068438    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-lbkvl\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="b5228372-7634-4439-ba4e-bc3fdf7463de" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:58.068536 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:58.068457    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:49:58.070396 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:49:58.070366    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-lbkvl\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="b5228372-7634-4439-ba4e-bc3fdf7463de" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:58.075929 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:58.075913    2569 scope.go:117] "RemoveContainer" containerID="3901a40c76812c845e0da0f7b1e8d8a48b0bdf906ccaa4bcdac3150abcbf674e"
Sep 11 16:49:58.076248 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:49:58.076227    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3901a40c76812c845e0da0f7b1e8d8a48b0bdf906ccaa4bcdac3150abcbf674e\": container with ID starting with 3901a40c76812c845e0da0f7b1e8d8a48b0bdf906ccaa4bcdac3150abcbf674e not found: ID does not exist" containerID="3901a40c76812c845e0da0f7b1e8d8a48b0bdf906ccaa4bcdac3150abcbf674e"
Sep 11 16:49:58.076320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:58.076260    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3901a40c76812c845e0da0f7b1e8d8a48b0bdf906ccaa4bcdac3150abcbf674e"} err="failed to get container status \"3901a40c76812c845e0da0f7b1e8d8a48b0bdf906ccaa4bcdac3150abcbf674e\": rpc error: code = NotFound desc = could not find container \"3901a40c76812c845e0da0f7b1e8d8a48b0bdf906ccaa4bcdac3150abcbf674e\": container with ID starting with 3901a40c76812c845e0da0f7b1e8d8a48b0bdf906ccaa4bcdac3150abcbf674e not found: ID does not exist"
Sep 11 16:49:58.095230 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:58.095179    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42" podStartSLOduration=2.110154439 podStartE2EDuration="5.09514066s" totalImagesPullingTime="2.984986221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:58.093932872 +0000 UTC m=+394.501151527" watchObservedRunningTime="2026-09-11 16:49:58.09514066 +0000 UTC m=+394.502359308"
Sep 11 16:49:58.096133 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:49:58.096095    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-lbkvl\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="b5228372-7634-4439-ba4e-bc3fdf7463de" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-lbkvl"
Sep 11 16:49:58.207134 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:49:58.207029    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5228372-7634-4439-ba4e-bc3fdf7463de" path="/var/lib/kubelet/pods/b5228372-7634-4439-ba4e-bc3fdf7463de/volumes"
Sep 11 16:50:00.221005 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:00.220969    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:00.282906 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:00.282877    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:00.418483 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:00.418448    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:00.523336 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:00.523250    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:04.028175 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:04.028143    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:04.129280 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:04.129240    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:06.056013 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:06.055980    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2rmt9"
Sep 11 16:50:09.076896 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:09.076863    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:50:09.827397 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:09.827310    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:09.917038 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:09.917007    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:10.023864 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:10.023829    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:14.195739 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:14.195702    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:14.290167 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:14.290135    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:22.178644 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:22.178611    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"]
Sep 11 16:50:22.179049 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:22.178823    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42" podUID="00334d37-7c5b-4927-8ada-623ed2f4136a" containerName="manager" containerID="cri-o://7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf" gracePeriod=10
Sep 11 16:50:22.425173 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:22.425146    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:50:22.504356 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:22.504266    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v4hs\" (UniqueName: \"kubernetes.io/projected/00334d37-7c5b-4927-8ada-623ed2f4136a-kube-api-access-6v4hs\") pod \"00334d37-7c5b-4927-8ada-623ed2f4136a\" (UID: \"00334d37-7c5b-4927-8ada-623ed2f4136a\") "
Sep 11 16:50:22.504356 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:22.504301    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00334d37-7c5b-4927-8ada-623ed2f4136a-extensions-socket-volume\") pod \"00334d37-7c5b-4927-8ada-623ed2f4136a\" (UID: \"00334d37-7c5b-4927-8ada-623ed2f4136a\") "
Sep 11 16:50:22.504791 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:22.504766    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00334d37-7c5b-4927-8ada-623ed2f4136a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "00334d37-7c5b-4927-8ada-623ed2f4136a" (UID: "00334d37-7c5b-4927-8ada-623ed2f4136a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:50:22.506536 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:22.506512    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00334d37-7c5b-4927-8ada-623ed2f4136a-kube-api-access-6v4hs" (OuterVolumeSpecName: "kube-api-access-6v4hs") pod "00334d37-7c5b-4927-8ada-623ed2f4136a" (UID: "00334d37-7c5b-4927-8ada-623ed2f4136a"). InnerVolumeSpecName "kube-api-access-6v4hs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:50:22.605419 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:22.605382    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v4hs\" (UniqueName: \"kubernetes.io/projected/00334d37-7c5b-4927-8ada-623ed2f4136a-kube-api-access-6v4hs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:50:22.605419 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:22.605412    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00334d37-7c5b-4927-8ada-623ed2f4136a-extensions-socket-volume\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:50:23.189004 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:23.188971    2569 generic.go:356] "Generic (PLEG): container finished" podID="00334d37-7c5b-4927-8ada-623ed2f4136a" containerID="7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf" exitCode=0
Sep 11 16:50:23.189463 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:23.189015    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"
Sep 11 16:50:23.189463 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:23.189051    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42" event={"ID":"00334d37-7c5b-4927-8ada-623ed2f4136a","Type":"ContainerDied","Data":"7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf"}
Sep 11 16:50:23.189463 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:23.189134    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42" event={"ID":"00334d37-7c5b-4927-8ada-623ed2f4136a","Type":"ContainerDied","Data":"18c41a1d00057a2759b31cda8f6a327bc5e3521245ddd8d524e16cf983394ba9"}
Sep 11 16:50:23.189463 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:23.189164    2569 scope.go:117] "RemoveContainer" containerID="7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf"
Sep 11 16:50:23.200145 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:23.200059    2569 scope.go:117] "RemoveContainer" containerID="7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf"
Sep 11 16:50:23.200656 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:50:23.200631    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf\": container with ID starting with 7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf not found: ID does not exist" containerID="7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf"
Sep 11 16:50:23.200736 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:23.200663    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf"} err="failed to get container status \"7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf\": rpc error: code = NotFound desc = could not find container \"7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf\": container with ID starting with 7618111550ed0d0d44f616728c61db16585020867ccf63abcd3abb651e142dbf not found: ID does not exist"
Sep 11 16:50:23.223188 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:23.223163    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"]
Sep 11 16:50:23.228276 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:23.228254    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-5mv42"]
Sep 11 16:50:24.205322 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:24.205288    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00334d37-7c5b-4927-8ada-623ed2f4136a" path="/var/lib/kubelet/pods/00334d37-7c5b-4927-8ada-623ed2f4136a/volumes"
Sep 11 16:50:25.545533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.545496    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"]
Sep 11 16:50:25.545907 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.545835    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5228372-7634-4439-ba4e-bc3fdf7463de" containerName="manager"
Sep 11 16:50:25.545907 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.545847    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5228372-7634-4439-ba4e-bc3fdf7463de" containerName="manager"
Sep 11 16:50:25.545907 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.545863    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00334d37-7c5b-4927-8ada-623ed2f4136a" containerName="manager"
Sep 11 16:50:25.545907 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.545869    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="00334d37-7c5b-4927-8ada-623ed2f4136a" containerName="manager"
Sep 11 16:50:25.546030 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.545929    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5228372-7634-4439-ba4e-bc3fdf7463de" containerName="manager"
Sep 11 16:50:25.546030 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.545938    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="00334d37-7c5b-4927-8ada-623ed2f4136a" containerName="manager"
Sep 11 16:50:25.550413 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.550395    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.553485 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.553465    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2jvxk\""
Sep 11 16:50:25.564949 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.564927    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"]
Sep 11 16:50:25.578571 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.578543    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:25.632047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.632016    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1006fc20-15e3-4325-9e4b-bc09745e2672-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.632229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.632065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.632229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.632115    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.632229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.632141    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.632229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.632157    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.632229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.632211    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r45gn\" (UniqueName: \"kubernetes.io/projected/1006fc20-15e3-4325-9e4b-bc09745e2672-kube-api-access-r45gn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.632413 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.632270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.632413 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.632301    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.632413 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.632322    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.733745 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.733700    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.733745 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.733749    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734007 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.733791    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1006fc20-15e3-4325-9e4b-bc09745e2672-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734007 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.733823    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734007 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.733839    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734007 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.733875    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734007 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.733901    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734007 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.733927    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r45gn\" (UniqueName: \"kubernetes.io/projected/1006fc20-15e3-4325-9e4b-bc09745e2672-kube-api-access-r45gn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734007 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.733956    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734381 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.734271    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734381 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.734350    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734458 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.734422    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734501 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.734477    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.734609 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.734594    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1006fc20-15e3-4325-9e4b-bc09745e2672-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.736450 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.736432    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.736611 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.736593    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.744522 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.744499    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1006fc20-15e3-4325-9e4b-bc09745e2672-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.745437 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.745417    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r45gn\" (UniqueName: \"kubernetes.io/projected/1006fc20-15e3-4325-9e4b-bc09745e2672-kube-api-access-r45gn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fq525\" (UID: \"1006fc20-15e3-4325-9e4b-bc09745e2672\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:25.814284 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.814209    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:25.860403 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:25.860368    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:26.138916 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:26.138887    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"]
Sep 11 16:50:26.141497 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:50:26.141471    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1006fc20_15e3_4325_9e4b_bc09745e2672.slice/crio-f656222b69149e4871833c32f3a5e57119cd582c1d5a4e0e8a61dab2415c7f1b WatchSource:0}: Error finding container f656222b69149e4871833c32f3a5e57119cd582c1d5a4e0e8a61dab2415c7f1b: Status 404 returned error can't find the container with id f656222b69149e4871833c32f3a5e57119cd582c1d5a4e0e8a61dab2415c7f1b
Sep 11 16:50:26.143600 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:26.143572    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:50:26.143672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:26.143633    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:50:26.143672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:26.143660    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:50:26.204852 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:26.204824    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525" event={"ID":"1006fc20-15e3-4325-9e4b-bc09745e2672","Type":"ContainerStarted","Data":"dd77d5903fd26cf3af851edf7442d495ec5ad4205b7b22a3b038ab9a7e4a4d6f"}
Sep 11 16:50:26.204961 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:26.204873    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525" event={"ID":"1006fc20-15e3-4325-9e4b-bc09745e2672","Type":"ContainerStarted","Data":"f656222b69149e4871833c32f3a5e57119cd582c1d5a4e0e8a61dab2415c7f1b"}
Sep 11 16:50:26.226062 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:26.225964    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525" podStartSLOduration=1.225942626 podStartE2EDuration="1.225942626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:50:26.225672342 +0000 UTC m=+422.632891013" watchObservedRunningTime="2026-09-11 16:50:26.225942626 +0000 UTC m=+422.633161276"
Sep 11 16:50:26.860833 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:26.860794    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:27.866595 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:27.866559    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:28.216535 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:28.216500    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:28.217514 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:28.217496    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:50:30.037370 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:30.037338    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:30.137580 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:30.137546    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:30.442423 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:30.442394    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:40.069524 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.069490    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:40.210420 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.210385    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:40.313611 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.313580    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:40.572323 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.572287    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:50:40.575334 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.575313    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"
Sep 11 16:50:40.577556 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.577522    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 16:50:40.577556 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.577524    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2xd5d\""
Sep 11 16:50:40.587290 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.587258    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:50:40.632781 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.632749    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:50:40.672308 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.672272    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgncw\" (UniqueName: \"kubernetes.io/projected/48573c38-144e-4ac3-8dd6-584af0fbc47f-kube-api-access-sgncw\") pod \"limitador-limitador-84994f7ffc-g5q5x\" (UID: \"48573c38-144e-4ac3-8dd6-584af0fbc47f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"
Sep 11 16:50:40.672492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.672332    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/48573c38-144e-4ac3-8dd6-584af0fbc47f-config-file\") pod \"limitador-limitador-84994f7ffc-g5q5x\" (UID: \"48573c38-144e-4ac3-8dd6-584af0fbc47f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"
Sep 11 16:50:40.773321 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.773285    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgncw\" (UniqueName: \"kubernetes.io/projected/48573c38-144e-4ac3-8dd6-584af0fbc47f-kube-api-access-sgncw\") pod \"limitador-limitador-84994f7ffc-g5q5x\" (UID: \"48573c38-144e-4ac3-8dd6-584af0fbc47f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"
Sep 11 16:50:40.773514 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.773337    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/48573c38-144e-4ac3-8dd6-584af0fbc47f-config-file\") pod \"limitador-limitador-84994f7ffc-g5q5x\" (UID: \"48573c38-144e-4ac3-8dd6-584af0fbc47f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"
Sep 11 16:50:40.773940 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.773922    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/48573c38-144e-4ac3-8dd6-584af0fbc47f-config-file\") pod \"limitador-limitador-84994f7ffc-g5q5x\" (UID: \"48573c38-144e-4ac3-8dd6-584af0fbc47f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"
Sep 11 16:50:40.790290 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.790267    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgncw\" (UniqueName: \"kubernetes.io/projected/48573c38-144e-4ac3-8dd6-584af0fbc47f-kube-api-access-sgncw\") pod \"limitador-limitador-84994f7ffc-g5q5x\" (UID: \"48573c38-144e-4ac3-8dd6-584af0fbc47f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"
Sep 11 16:50:40.887295 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:40.887264    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"
Sep 11 16:50:41.075067 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:41.075022    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:41.142734 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:41.142637    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:41.164053 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:41.164027    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:50:41.166848 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:50:41.166817    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48573c38_144e_4ac3_8dd6_584af0fbc47f.slice/crio-0a611ad40728848df4d8e4fd70a4ec0f1ded18c286c0a8915e7c7be7b9042593 WatchSource:0}: Error finding container 0a611ad40728848df4d8e4fd70a4ec0f1ded18c286c0a8915e7c7be7b9042593: Status 404 returned error can't find the container with id 0a611ad40728848df4d8e4fd70a4ec0f1ded18c286c0a8915e7c7be7b9042593
Sep 11 16:50:41.276293 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:41.276261    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x" event={"ID":"48573c38-144e-4ac3-8dd6-584af0fbc47f","Type":"ContainerStarted","Data":"0a611ad40728848df4d8e4fd70a4ec0f1ded18c286c0a8915e7c7be7b9042593"}
Sep 11 16:50:42.282782 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:42.282752    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x" event={"ID":"48573c38-144e-4ac3-8dd6-584af0fbc47f","Type":"ContainerStarted","Data":"f184eea8b83bf03e6c8e2e08ed2532185a9d8c43b3d74ef1a15bf00fa51d2a35"}
Sep 11 16:50:42.283211 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:42.282877    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"
Sep 11 16:50:42.301516 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:42.301458    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x" podStartSLOduration=1.40903822 podStartE2EDuration="2.301438709s" totalImagesPullingTime="892.400489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:50:42.301261272 +0000 UTC m=+438.708479939" watchObservedRunningTime="2026-09-11 16:50:42.301438709 +0000 UTC m=+438.708657356"
Sep 11 16:50:53.289054 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:53.288985    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"
Sep 11 16:50:58.800818 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:58.800767    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:58.902733 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:58.902702    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:50:59.360840 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:50:59.360803    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:08.628231 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:08.628195    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:09.019823 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.019746    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:09.183369 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.183325    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65d45d6fb7-9z5hd"]
Sep 11 16:51:09.186989 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.186968    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d45d6fb7-9z5hd"
Sep 11 16:51:09.189783 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.189761    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 16:51:09.189783 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.189776    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rdt2v\""
Sep 11 16:51:09.196297 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.196271    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65d45d6fb7-9z5hd"]
Sep 11 16:51:09.334827 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.334719    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8kh9\" (UniqueName: \"kubernetes.io/projected/498efbd2-ab64-47ea-befb-2ce5d4ba3790-kube-api-access-g8kh9\") pod \"authorino-65d45d6fb7-9z5hd\" (UID: \"498efbd2-ab64-47ea-befb-2ce5d4ba3790\") " pod="kuadrant-system/authorino-65d45d6fb7-9z5hd"
Sep 11 16:51:09.334827 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.334784    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/498efbd2-ab64-47ea-befb-2ce5d4ba3790-tls-cert\") pod \"authorino-65d45d6fb7-9z5hd\" (UID: \"498efbd2-ab64-47ea-befb-2ce5d4ba3790\") " pod="kuadrant-system/authorino-65d45d6fb7-9z5hd"
Sep 11 16:51:09.435879 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.435838    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8kh9\" (UniqueName: \"kubernetes.io/projected/498efbd2-ab64-47ea-befb-2ce5d4ba3790-kube-api-access-g8kh9\") pod \"authorino-65d45d6fb7-9z5hd\" (UID: \"498efbd2-ab64-47ea-befb-2ce5d4ba3790\") " pod="kuadrant-system/authorino-65d45d6fb7-9z5hd"
Sep 11 16:51:09.435879 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.435884    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/498efbd2-ab64-47ea-befb-2ce5d4ba3790-tls-cert\") pod \"authorino-65d45d6fb7-9z5hd\" (UID: \"498efbd2-ab64-47ea-befb-2ce5d4ba3790\") " pod="kuadrant-system/authorino-65d45d6fb7-9z5hd"
Sep 11 16:51:09.438428 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.438399    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/498efbd2-ab64-47ea-befb-2ce5d4ba3790-tls-cert\") pod \"authorino-65d45d6fb7-9z5hd\" (UID: \"498efbd2-ab64-47ea-befb-2ce5d4ba3790\") " pod="kuadrant-system/authorino-65d45d6fb7-9z5hd"
Sep 11 16:51:09.446011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.445986    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8kh9\" (UniqueName: \"kubernetes.io/projected/498efbd2-ab64-47ea-befb-2ce5d4ba3790-kube-api-access-g8kh9\") pod \"authorino-65d45d6fb7-9z5hd\" (UID: \"498efbd2-ab64-47ea-befb-2ce5d4ba3790\") " pod="kuadrant-system/authorino-65d45d6fb7-9z5hd"
Sep 11 16:51:09.496103 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.496040    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d45d6fb7-9z5hd"
Sep 11 16:51:09.775934 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:09.775896    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65d45d6fb7-9z5hd"]
Sep 11 16:51:09.779352 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:51:09.779323    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod498efbd2_ab64_47ea_befb_2ce5d4ba3790.slice/crio-773cdae63ff40e2bc880ce9c616b6d7b92d1e3d5933423813ac88226e40ea704 WatchSource:0}: Error finding container 773cdae63ff40e2bc880ce9c616b6d7b92d1e3d5933423813ac88226e40ea704: Status 404 returned error can't find the container with id 773cdae63ff40e2bc880ce9c616b6d7b92d1e3d5933423813ac88226e40ea704
Sep 11 16:51:10.416806 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:10.416763    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d45d6fb7-9z5hd" event={"ID":"498efbd2-ab64-47ea-befb-2ce5d4ba3790","Type":"ContainerStarted","Data":"773cdae63ff40e2bc880ce9c616b6d7b92d1e3d5933423813ac88226e40ea704"}
Sep 11 16:51:11.422845 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:11.422808    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d45d6fb7-9z5hd" event={"ID":"498efbd2-ab64-47ea-befb-2ce5d4ba3790","Type":"ContainerStarted","Data":"3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4"}
Sep 11 16:51:11.442862 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:11.442759    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65d45d6fb7-9z5hd" podStartSLOduration=1.044756058 podStartE2EDuration="2.442740402s" totalImagesPullingTime="1.397984344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:51:11.440891986 +0000 UTC m=+467.848110645" watchObservedRunningTime="2026-09-11 16:51:11.442740402 +0000 UTC m=+467.849959054"
Sep 11 16:51:16.454828 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:16.454790    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:16.533381 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:16.533347    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:16.595723 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:16.595691    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:16.690888 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:16.690852    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:16.750432 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:16.750341    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:19.740675 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:19.740637    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:19.830481 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:19.830451    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:24.000084 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:24.000052    2569 scope.go:117] "RemoveContainer" containerID="2b6442564c7713e7cc380b6a085bbf3bd637961da4efc16718bf0963af78434e"
Sep 11 16:51:34.430998 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:34.430961    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:34.537457 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:34.537422    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:53.251222 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:53.251192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:53.356464 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:53.356404    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:54.630291 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:54.630259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:54.729118 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:54.729066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:54.987417 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:54.987296    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:55.080669 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:55.080637    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:55.351020 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:55.350932    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:55.444692 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:55.444646    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:51:55.498760 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:51:55.498722    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:52:04.682467 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:04.682430    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:52:04.692267 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:04.692240    2569 generic.go:356] "Generic (PLEG): container finished" podID="1006fc20-15e3-4325-9e4b-bc09745e2672" containerID="dd77d5903fd26cf3af851edf7442d495ec5ad4205b7b22a3b038ab9a7e4a4d6f" exitCode=0
Sep 11 16:52:04.692415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:04.692314    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525" event={"ID":"1006fc20-15e3-4325-9e4b-bc09745e2672","Type":"ContainerDied","Data":"dd77d5903fd26cf3af851edf7442d495ec5ad4205b7b22a3b038ab9a7e4a4d6f"}
Sep 11 16:52:04.692671 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:04.692659    2569 scope.go:117] "RemoveContainer" containerID="dd77d5903fd26cf3af851edf7442d495ec5ad4205b7b22a3b038ab9a7e4a4d6f"
Sep 11 16:52:04.693665 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:04.693639    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:52:04.693724 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:04.693707    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:52:04.693774 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:04.693740    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:52:04.777460 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:04.777423    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:52:05.698471 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:05.698438    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525" event={"ID":"1006fc20-15e3-4325-9e4b-bc09745e2672","Type":"ContainerStarted","Data":"ef8bc71a6b99a66c444bed33ff3785a8602130a316bc56d1230923fb76291392"}
Sep 11 16:52:05.861025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:05.860985    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:52:05.861292 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:05.861256    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fq525 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 16:52:05.861387 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:05.861337    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525" podUID="1006fc20-15e3-4325-9e4b-bc09745e2672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 16:52:06.860829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:06.860792    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fq525 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 16:52:06.861240 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:06.860862    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525" podUID="1006fc20-15e3-4325-9e4b-bc09745e2672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 16:52:07.861302 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:07.861259    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fq525 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 16:52:07.861697 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:07.861320    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525" podUID="1006fc20-15e3-4325-9e4b-bc09745e2672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 16:52:08.861786 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:08.861743    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fq525 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 16:52:08.862214 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:08.861821    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525" podUID="1006fc20-15e3-4325-9e4b-bc09745e2672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 16:52:09.860858 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:09.860820    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fq525 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 16:52:09.861041 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:09.860883    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525" podUID="1006fc20-15e3-4325-9e4b-bc09745e2672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 16:52:10.864736 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:10.864704    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:52:10.865246 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:10.865099    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:52:10.865698 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:10.865680    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fq525"
Sep 11 16:52:10.869096 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:10.869064    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:52:11.500967 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:11.500924    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:52:16.032277 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:16.032246    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:52:16.688349 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:16.688312    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:52:25.325110 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:25.325057    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:52:51.646874 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:51.646836    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:52:52.688962 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:52.688922    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:52:54.011833 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.011799    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"]
Sep 11 16:52:54.015195 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.015178    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.025530 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.025505    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 16:52:54.025661 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.025542    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 11 16:52:54.025661 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.025556    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 16:52:54.025661 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.025609    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vkjjm\""
Sep 11 16:52:54.051723 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.051691    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"]
Sep 11 16:52:54.079916 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.079878    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.080119 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.079923    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.080119 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.079951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnrf2\" (UniqueName: \"kubernetes.io/projected/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-kube-api-access-jnrf2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.080260 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.080106    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.080260 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.080145    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.080260 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.080227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.132190 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.132158    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:52:54.181278 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.181247    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.181445 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.181288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnrf2\" (UniqueName: \"kubernetes.io/projected/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-kube-api-access-jnrf2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.181445 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.181338    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.181445 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.181353    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.181445 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.181400    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.181660 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.181445    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.181719 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.181663    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.181765 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.181722    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.181841 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.181826    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.183624 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.183606    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.183900 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.183883    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.189353 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.189330    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnrf2\" (UniqueName: \"kubernetes.io/projected/8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a-kube-api-access-jnrf2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whg6n\" (UID: \"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.325233 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.325139    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:52:54.608680 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.608655    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"]
Sep 11 16:52:54.610557 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:52:54.610531    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bffcfb2_7c5b_4921_9f1f_c76a7b3a501a.slice/crio-ba315a1252443bd2a1980e6793becb313908ffe2faf2f852e84bb80ec4583372 WatchSource:0}: Error finding container ba315a1252443bd2a1980e6793becb313908ffe2faf2f852e84bb80ec4583372: Status 404 returned error can't find the container with id ba315a1252443bd2a1980e6793becb313908ffe2faf2f852e84bb80ec4583372
Sep 11 16:52:54.935440 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:54.935395    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n" event={"ID":"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a","Type":"ContainerStarted","Data":"ba315a1252443bd2a1980e6793becb313908ffe2faf2f852e84bb80ec4583372"}
Sep 11 16:52:55.943584 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:55.943489    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n" event={"ID":"8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a","Type":"ContainerStarted","Data":"1fa503959e867848ed7b0eecfd3904022b75b43f7725830420bd5b5cd757e8f0"}
Sep 11 16:52:55.963782 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:55.963718    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n" podStartSLOduration=1.931605943 podStartE2EDuration="2.963697636s" totalImagesPullingTime="1.032091693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:52:55.962962602 +0000 UTC m=+572.370181262" watchObservedRunningTime="2026-09-11 16:52:55.963697636 +0000 UTC m=+572.370916285"
Sep 11 16:52:57.779674 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:52:57.779637    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:53:00.683520 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:00.683484    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:53:00.763345 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:00.763312    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:53:04.326101 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:04.326051    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:53:04.326547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:04.326120    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:53:04.339236 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:04.339203    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:53:04.999171 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:04.999137    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whg6n"
Sep 11 16:53:05.051921 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:05.051889    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:53:08.223878 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:08.223839    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:53:09.792709 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:09.792660    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:53:12.545684 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.545636    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"]
Sep 11 16:53:12.551561 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.551536    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.556469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.556446    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 11 16:53:12.565285 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.565260    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"]
Sep 11 16:53:12.606177 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.606144    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:53:12.659002 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.658971    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.659002 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.659002    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.659217 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.659096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.659217 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.659129    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.659217 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.659175    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjpvk\" (UniqueName: \"kubernetes.io/projected/17ffca60-3170-4fad-a29f-235be9ea7ae1-kube-api-access-rjpvk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.659316 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.659237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17ffca60-3170-4fad-a29f-235be9ea7ae1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.760479 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.760442    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.760479 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.760484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.760749 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.760540    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.760749 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.760560    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.760749 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.760613    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjpvk\" (UniqueName: \"kubernetes.io/projected/17ffca60-3170-4fad-a29f-235be9ea7ae1-kube-api-access-rjpvk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.760749 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.760681    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17ffca60-3170-4fad-a29f-235be9ea7ae1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.761016 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.760993    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.761016 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.761010    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.761180 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.761066    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.762798 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.762777    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17ffca60-3170-4fad-a29f-235be9ea7ae1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.763186 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.763166    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17ffca60-3170-4fad-a29f-235be9ea7ae1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.769330 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.769312    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjpvk\" (UniqueName: \"kubernetes.io/projected/17ffca60-3170-4fad-a29f-235be9ea7ae1-kube-api-access-rjpvk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2cwf9\" (UID: \"17ffca60-3170-4fad-a29f-235be9ea7ae1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:12.866587 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:12.866484    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:13.155981 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:13.155920    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"]
Sep 11 16:53:13.158302 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:53:13.158273    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ffca60_3170_4fad_a29f_235be9ea7ae1.slice/crio-8c3100521940340982cc615f1c1d2102c73e658217f338d9c7cf1d5bc952d84f WatchSource:0}: Error finding container 8c3100521940340982cc615f1c1d2102c73e658217f338d9c7cf1d5bc952d84f: Status 404 returned error can't find the container with id 8c3100521940340982cc615f1c1d2102c73e658217f338d9c7cf1d5bc952d84f
Sep 11 16:53:14.038379 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:14.038344    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9" event={"ID":"17ffca60-3170-4fad-a29f-235be9ea7ae1","Type":"ContainerStarted","Data":"7c0e39ecd17db52a97d14c65fbad8c4749b91086be2a2d487b8beded05e1bda9"}
Sep 11 16:53:14.038980 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:14.038436    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9" event={"ID":"17ffca60-3170-4fad-a29f-235be9ea7ae1","Type":"ContainerStarted","Data":"8c3100521940340982cc615f1c1d2102c73e658217f338d9c7cf1d5bc952d84f"}
Sep 11 16:53:14.059888 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:14.059834    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9" podStartSLOduration=1.856991999 podStartE2EDuration="2.059817165s" totalImagesPullingTime="202.825166ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:53:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:53:14.058695575 +0000 UTC m=+590.465914240" watchObservedRunningTime="2026-09-11 16:53:14.059817165 +0000 UTC m=+590.467035812"
Sep 11 16:53:14.797450 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:14.797408    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:53:20.741700 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:20.741659    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:53:22.867321 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:22.867285    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:22.867766 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:22.867345    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:22.880089 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:22.880052    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:22.992568 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:22.992536    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:53:23.096766 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:23.096735    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2cwf9"
Sep 11 16:53:42.961386 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:42.961352    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:53:43.062449 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:43.062414    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:53:54.037211 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:53:54.037135    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:12.726838 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:12.726798    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65d45d6fb7-9z5hd"]
Sep 11 16:55:12.727383 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:12.726971    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-65d45d6fb7-9z5hd" podUID="498efbd2-ab64-47ea-befb-2ce5d4ba3790" containerName="authorino" containerID="cri-o://3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4" gracePeriod=30
Sep 11 16:55:12.981803 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:12.981739    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d45d6fb7-9z5hd"
Sep 11 16:55:13.063243 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.063213    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8kh9\" (UniqueName: \"kubernetes.io/projected/498efbd2-ab64-47ea-befb-2ce5d4ba3790-kube-api-access-g8kh9\") pod \"498efbd2-ab64-47ea-befb-2ce5d4ba3790\" (UID: \"498efbd2-ab64-47ea-befb-2ce5d4ba3790\") "
Sep 11 16:55:13.063428 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.063317    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/498efbd2-ab64-47ea-befb-2ce5d4ba3790-tls-cert\") pod \"498efbd2-ab64-47ea-befb-2ce5d4ba3790\" (UID: \"498efbd2-ab64-47ea-befb-2ce5d4ba3790\") "
Sep 11 16:55:13.065463 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.065431    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/498efbd2-ab64-47ea-befb-2ce5d4ba3790-kube-api-access-g8kh9" (OuterVolumeSpecName: "kube-api-access-g8kh9") pod "498efbd2-ab64-47ea-befb-2ce5d4ba3790" (UID: "498efbd2-ab64-47ea-befb-2ce5d4ba3790"). InnerVolumeSpecName "kube-api-access-g8kh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:55:13.073522 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.073492    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/498efbd2-ab64-47ea-befb-2ce5d4ba3790-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "498efbd2-ab64-47ea-befb-2ce5d4ba3790" (UID: "498efbd2-ab64-47ea-befb-2ce5d4ba3790"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:55:13.163957 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.163923    2569 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/498efbd2-ab64-47ea-befb-2ce5d4ba3790-tls-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:55:13.163957 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.163952    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8kh9\" (UniqueName: \"kubernetes.io/projected/498efbd2-ab64-47ea-befb-2ce5d4ba3790-kube-api-access-g8kh9\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:55:13.403505 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.403469    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:13.499306 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.499273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:13.617896 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.617864    2569 generic.go:356] "Generic (PLEG): container finished" podID="498efbd2-ab64-47ea-befb-2ce5d4ba3790" containerID="3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4" exitCode=0
Sep 11 16:55:13.618110 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.617904    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d45d6fb7-9z5hd"
Sep 11 16:55:13.618110 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.617958    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d45d6fb7-9z5hd" event={"ID":"498efbd2-ab64-47ea-befb-2ce5d4ba3790","Type":"ContainerDied","Data":"3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4"}
Sep 11 16:55:13.618110 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.618013    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d45d6fb7-9z5hd" event={"ID":"498efbd2-ab64-47ea-befb-2ce5d4ba3790","Type":"ContainerDied","Data":"773cdae63ff40e2bc880ce9c616b6d7b92d1e3d5933423813ac88226e40ea704"}
Sep 11 16:55:13.618110 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.618036    2569 scope.go:117] "RemoveContainer" containerID="3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4"
Sep 11 16:55:13.627794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.627775    2569 scope.go:117] "RemoveContainer" containerID="3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4"
Sep 11 16:55:13.628127 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:55:13.628110    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4\": container with ID starting with 3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4 not found: ID does not exist" containerID="3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4"
Sep 11 16:55:13.628183 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.628140    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4"} err="failed to get container status \"3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4\": rpc error: code = NotFound desc = could not find container \"3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4\": container with ID starting with 3a0f2e16e6588d2f8714600e0f0f7db931b5318890069ff23a4d7b48660f5fb4 not found: ID does not exist"
Sep 11 16:55:13.682381 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.682295    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65d45d6fb7-9z5hd"]
Sep 11 16:55:13.717768 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:13.717731    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-65d45d6fb7-9z5hd"]
Sep 11 16:55:14.204907 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:14.204874    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="498efbd2-ab64-47ea-befb-2ce5d4ba3790" path="/var/lib/kubelet/pods/498efbd2-ab64-47ea-befb-2ce5d4ba3790/volumes"
Sep 11 16:55:14.691055 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:14.691025    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:14.785762 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:14.785723    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:30.956819 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:30.956761    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:31.061087 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.061039    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:31.162035 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.161998    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:31.323740 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.323661    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"]
Sep 11 16:55:31.324051 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.324039    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="498efbd2-ab64-47ea-befb-2ce5d4ba3790" containerName="authorino"
Sep 11 16:55:31.324118 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.324052    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="498efbd2-ab64-47ea-befb-2ce5d4ba3790" containerName="authorino"
Sep 11 16:55:31.324175 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.324141    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="498efbd2-ab64-47ea-befb-2ce5d4ba3790" containerName="authorino"
Sep 11 16:55:31.328402 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.328380    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.332118 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.332092    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-2dc95c-openshift-default-dockercfg-bgp8m\""
Sep 11 16:55:31.347586 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.347555    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"]
Sep 11 16:55:31.393827 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.393792    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:31.439775 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.439744    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-certs\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.439969 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.439810    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-data\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.439969 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.439849    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlpp7\" (UniqueName: \"kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-kube-api-access-nlpp7\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.439969 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.439882    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-socket\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.439969 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.439911    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-token\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.440184 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.439981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-envoy\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.440184 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.440025    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istiod-ca-cert\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.440184 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.440143    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-credential-socket\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.440289 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.440186    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-podinfo\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.461715 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.461680    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"]
Sep 11 16:55:31.465307 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.465288    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.468995 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.468961    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4268e3-openshift-default-dockercfg-dxb59\""
Sep 11 16:55:31.481417 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.481386    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"]
Sep 11 16:55:31.516615 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.516568    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:31.540851 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.540819    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-credential-socket\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.540868    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-podinfo\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.540894    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-certs\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.540915    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-data\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.540937    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlpp7\" (UniqueName: \"kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-kube-api-access-nlpp7\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.540961    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-socket\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.540982    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-token\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.541001    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-envoy\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.541038    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istiod-ca-cert\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541568 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.541536    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-certs\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541568 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.541557    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-credential-socket\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541715 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.541580    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-data\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541715 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.541651    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-socket\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.541902 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.541883    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istiod-ca-cert\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.543587 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.543567    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-envoy\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.543674 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.543608    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-podinfo\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.567529 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.567501    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlpp7\" (UniqueName: \"kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-kube-api-access-nlpp7\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.569070 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.569044    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"]
Sep 11 16:55:31.569907 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.569884    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-token\") pod \"e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") " pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.574304 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.574250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.577839 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.577817    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-52b0af-openshift-default-dockercfg-f85t6\""
Sep 11 16:55:31.589352 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.589326    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"]
Sep 11 16:55:31.624582 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.624552    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:31.639633 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.639601    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:31.641971 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.641947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-token\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.642066 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.641986    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-certs\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.642066 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.642005    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istiod-ca-cert\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.642209 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.642158    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-socket\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.642209 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.642196    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-data\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.642318 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.642239    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-credential-socket\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.642318 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.642265    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-envoy\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.642318 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.642303    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-podinfo\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.642455 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.642330    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htrhj\" (UniqueName: \"kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-kube-api-access-htrhj\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.743292 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743250    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-token\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.743292 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743291    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-socket\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.743555 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-certs\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.743555 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743339    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istiod-ca-cert\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.743555 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743402    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-socket\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.743555 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743454    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-credential-socket\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.743555 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743491    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-token\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.743555 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743519    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-credential-socket\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.743936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743596    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-envoy\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.743936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743687    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-podinfo\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.743936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743741    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-envoy\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.743936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743759    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-certs\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.743936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743767    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-data\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.743936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743851    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7dz9\" (UniqueName: \"kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-kube-api-access-g7dz9\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.743936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743868    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-credential-socket\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.743936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743883    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istiod-ca-cert\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.743936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743923    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-socket\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.744410 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743951    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-data\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.744410 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.743989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-certs\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.744410 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.744024    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htrhj\" (UniqueName: \"kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-kube-api-access-htrhj\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.744410 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.744063    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-podinfo\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.744410 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.744148    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istiod-ca-cert\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.744410 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.744278    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-data\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.745847 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.745820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-envoy\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.746488 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.746472    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-podinfo\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.764919 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.764891    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htrhj\" (UniqueName: \"kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-kube-api-access-htrhj\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.770063 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.770041    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-token\") pod \"e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") " pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.776064 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.776041    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:31.787116 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.787059    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:31.844843 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.844806    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-podinfo\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.845037 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.844887    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-envoy\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.845037 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.844914    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-data\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.845037 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.844958    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7dz9\" (UniqueName: \"kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-kube-api-access-g7dz9\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.845037 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.844989    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istiod-ca-cert\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.845298 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.845039    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-certs\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.845298 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.845134    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-socket\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.845298 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.845200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-credential-socket\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.845298 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.845232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-token\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.845510 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.845448    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-data\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.846181 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.845617    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-certs\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.846181 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.845701    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-socket\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.846181 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.846128    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-credential-socket\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.846480 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.846414    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istiod-ca-cert\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.847765 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.847743    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-podinfo\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.847886 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.847870    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-envoy\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.863349 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.863320    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7dz9\" (UniqueName: \"kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-kube-api-access-g7dz9\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.874700 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.874672    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-token\") pod \"e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") " pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.895927 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.895884    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:31.949439 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.949343    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"]
Sep 11 16:55:31.951863 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:55:31.951826    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3bf67c2_4778_4df3_acc5_e64b3282c37f.slice/crio-5bd2b6232c3ef692f159101fa02e87f48508dbffa69d23571333121141e4a272 WatchSource:0}: Error finding container 5bd2b6232c3ef692f159101fa02e87f48508dbffa69d23571333121141e4a272: Status 404 returned error can't find the container with id 5bd2b6232c3ef692f159101fa02e87f48508dbffa69d23571333121141e4a272
Sep 11 16:55:31.955887 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.955104    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:31.955887 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.955203    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:31.955887 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:31.955245    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:32.012703 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.012675    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:32.079533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.079501    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"]
Sep 11 16:55:32.081332 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:55:32.081297    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3de30f_d85a_4e9d_94e6_35a490a3678b.slice/crio-addeb1fbe6196668d05d8a4b3deab620c93b4c7f5ec1409f5705b3accc393891 WatchSource:0}: Error finding container addeb1fbe6196668d05d8a4b3deab620c93b4c7f5ec1409f5705b3accc393891: Status 404 returned error can't find the container with id addeb1fbe6196668d05d8a4b3deab620c93b4c7f5ec1409f5705b3accc393891
Sep 11 16:55:32.083876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.083841    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:32.083944 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.083923    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:32.083986 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.083961    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:32.150538 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.150483    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:32.183472 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.183430    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"]
Sep 11 16:55:32.186710 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:55:32.186678    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70eb72e_2b80_42b5_b6ed_ac6b9a01dc91.slice/crio-c79057d369f2c1e2247e13111979c0d3b37f4ca94d893a30d13ea74ee86b2c45 WatchSource:0}: Error finding container c79057d369f2c1e2247e13111979c0d3b37f4ca94d893a30d13ea74ee86b2c45: Status 404 returned error can't find the container with id c79057d369f2c1e2247e13111979c0d3b37f4ca94d893a30d13ea74ee86b2c45
Sep 11 16:55:32.189803 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.189766    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:32.189887 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.189853    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:32.189947 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.189932    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:32.527224 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.527194    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:32.655648 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.655608    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:32.716783 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.716747    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" event={"ID":"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91","Type":"ContainerStarted","Data":"31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a"}
Sep 11 16:55:32.717003 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.716810    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" event={"ID":"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91","Type":"ContainerStarted","Data":"c79057d369f2c1e2247e13111979c0d3b37f4ca94d893a30d13ea74ee86b2c45"}
Sep 11 16:55:32.718482 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.718455    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" event={"ID":"1d3de30f-d85a-4e9d-94e6-35a490a3678b","Type":"ContainerStarted","Data":"0cce37555d791bf6e2e5f44c3c471583712cf0750b30137419c3f2c76c72a4c5"}
Sep 11 16:55:32.718592 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.718500    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" event={"ID":"1d3de30f-d85a-4e9d-94e6-35a490a3678b","Type":"ContainerStarted","Data":"addeb1fbe6196668d05d8a4b3deab620c93b4c7f5ec1409f5705b3accc393891"}
Sep 11 16:55:32.720085 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.720050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" event={"ID":"d3bf67c2-4778-4df3-acc5-e64b3282c37f","Type":"ContainerStarted","Data":"703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15"}
Sep 11 16:55:32.720191 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.720128    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" event={"ID":"d3bf67c2-4778-4df3-acc5-e64b3282c37f","Type":"ContainerStarted","Data":"5bd2b6232c3ef692f159101fa02e87f48508dbffa69d23571333121141e4a272"}
Sep 11 16:55:32.749879 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.749820    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" podStartSLOduration=1.749801248 podStartE2EDuration="1.749801248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:55:32.747625637 +0000 UTC m=+729.154844284" watchObservedRunningTime="2026-09-11 16:55:32.749801248 +0000 UTC m=+729.157019895"
Sep 11 16:55:32.774535 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.774479    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" podStartSLOduration=1.774459544 podStartE2EDuration="1.774459544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:55:32.771800217 +0000 UTC m=+729.179018884" watchObservedRunningTime="2026-09-11 16:55:32.774459544 +0000 UTC m=+729.181678195"
Sep 11 16:55:32.777116 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.777092    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:32.801791 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.801676    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" podStartSLOduration=1.8016591979999999 podStartE2EDuration="1.801659198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:55:32.79867143 +0000 UTC m=+729.205890108" watchObservedRunningTime="2026-09-11 16:55:32.801659198 +0000 UTC m=+729.208877845"
Sep 11 16:55:32.896037 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.895999    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:32.969597 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:32.969554    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:33.639822 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:33.639787    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:33.644707 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:33.644681    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:33.727892 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:33.727863    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:33.728931 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:33.728908    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:33.782852 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:33.782823    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:33.901978 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:33.901906    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:34.732629 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:34.732599    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:34.733048 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:34.732685    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:34.733421 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:34.733405    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:34.733495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:34.733456    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:34.841019 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:34.840979    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:34.919876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:34.919845    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:55:35.071377 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:35.071291    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:35.303786 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:35.303760    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:36.669859 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:36.669827    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:36.813490 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:36.813460    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:37.297278 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:37.297222    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:37.425316 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:37.425286    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:38.001598 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:38.001564    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:38.132130 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:38.132066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:38.283037 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:38.282950    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:38.491199 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:38.491151    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:38.610047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:38.609958    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:39.072762 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:39.072723    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:39.203167 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:39.203123    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:39.376657 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:39.376631    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:39.630961 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:39.630854    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:39.736350 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:39.736295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:40.014714 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.014622    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:40.118370 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.118337    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:40.313373 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.313295    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"]
Sep 11 16:55:40.317210 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.317185    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.320063 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.320038    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-8b1a16-openshift-default-dockercfg-qrjch\""
Sep 11 16:55:40.335626 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.335598    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"]
Sep 11 16:55:40.368508 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.368444    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:40.432384 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.432344    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-certs\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.432596 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.432411    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-socket\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.432596 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.432480    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-token\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.432719 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.432547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-envoy\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.432719 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.432689    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-podinfo\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.432823 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.432720    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxs4s\" (UniqueName: \"kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-kube-api-access-gxs4s\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.432823 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.432762    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-data\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.432823 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.432778    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15380047-35b2-4d02-8bb4-81a0c09a6519-istiod-ca-cert\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.432823 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.432806    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-credential-socket\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.533525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.533493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-podinfo\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.533747 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.533534    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxs4s\" (UniqueName: \"kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-kube-api-access-gxs4s\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.533747 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.533563    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-data\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.533747 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.533578    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15380047-35b2-4d02-8bb4-81a0c09a6519-istiod-ca-cert\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.533747 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.533708    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-credential-socket\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.533971 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.533830    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-certs\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.533971 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.533884    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-socket\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.533971 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.533924    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-token\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.534171 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.533978    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-envoy\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.534171 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.534059    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-data\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.534249 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.534189    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-credential-socket\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.534284 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.534262    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-socket\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.534455 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.534435    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-certs\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.534598 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.534582    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15380047-35b2-4d02-8bb4-81a0c09a6519-istiod-ca-cert\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.536233 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.536208    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-podinfo\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.536330 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.536282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-envoy\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.542640 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.542607    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-token\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.542757 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.542692    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxs4s\" (UniqueName: \"kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-kube-api-access-gxs4s\") pod \"e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") " pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.629949 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.629916    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:40.842418 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.842384    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:40.917810 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.917790    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"]
Sep 11 16:55:40.920211 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:55:40.920180    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15380047_35b2_4d02_8bb4_81a0c09a6519.slice/crio-01f3b609962fb444b4d375a253bca36756555f20575d477748190b78afce96c8 WatchSource:0}: Error finding container 01f3b609962fb444b4d375a253bca36756555f20575d477748190b78afce96c8: Status 404 returned error can't find the container with id 01f3b609962fb444b4d375a253bca36756555f20575d477748190b78afce96c8
Sep 11 16:55:40.922571 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.922539    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:40.922657 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.922617    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:40.922657 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:40.922647    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:41.085492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:41.085460    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:41.769455 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:41.769412    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" event={"ID":"15380047-35b2-4d02-8bb4-81a0c09a6519","Type":"ContainerStarted","Data":"283931ece6996db539885cdc09f0d7ef887983e61b9f33d107a924f46e0382d0"}
Sep 11 16:55:41.769843 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:41.769477    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" event={"ID":"15380047-35b2-4d02-8bb4-81a0c09a6519","Type":"ContainerStarted","Data":"01f3b609962fb444b4d375a253bca36756555f20575d477748190b78afce96c8"}
Sep 11 16:55:41.793054 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:41.792988    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" podStartSLOduration=1.792968052 podStartE2EDuration="1.792968052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:55:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:55:41.791408731 +0000 UTC m=+738.198627372" watchObservedRunningTime="2026-09-11 16:55:41.792968052 +0000 UTC m=+738.200186769"
Sep 11 16:55:42.010715 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.010678    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:42.191200 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.191166    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:42.296182 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.296149    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:42.445912 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.445818    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:42.630827 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.630790    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:42.635857 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.635829    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:42.752564 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.752481    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"]
Sep 11 16:55:42.756414 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.756393    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.760311 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.760287    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ab1ade68-openshift-default-dockercfg-4d2rw\""
Sep 11 16:55:42.772189 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.772156    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"]
Sep 11 16:55:42.776278 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.776242    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:42.777189 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.777169    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:55:42.825640 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.825604    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:42.861848 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.861808    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4f14aa-2055-4c78-9549-0d905baa5723-istiod-ca-cert\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.862117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.862060    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-envoy\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.862239 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.862224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-podinfo\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.863013 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.862986    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-token\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.863307 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.863286    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-data\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.863435 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.863419    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-certs\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.863544 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.863530    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-socket\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.863660 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.863646    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-credential-socket\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.863839 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.863824    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpsw9\" (UniqueName: \"kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-kube-api-access-mpsw9\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.965481 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.965424    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-data\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.965481 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.965483    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-certs\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.965778 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.965505    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-socket\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.965778 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.965532    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-credential-socket\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.965778 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.965550    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpsw9\" (UniqueName: \"kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-kube-api-access-mpsw9\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.965778 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.965634    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4f14aa-2055-4c78-9549-0d905baa5723-istiod-ca-cert\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.965778 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.965678    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-envoy\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.965778 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.965702    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-podinfo\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.965778 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.965761    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-token\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.966142 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.965912    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-data\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.966142 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.965962    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-socket\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.966142 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.966019    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-certs\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.966246 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.966150    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-credential-socket\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.966385 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.966362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4f14aa-2055-4c78-9549-0d905baa5723-istiod-ca-cert\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.968102 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.968053    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-envoy\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.968296 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.968279    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-podinfo\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.978944 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.978917    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpsw9\" (UniqueName: \"kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-kube-api-access-mpsw9\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:42.982761 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:42.982733    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-token\") pod \"e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") " pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:43.067788 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:43.067645    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:43.206489 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:43.206413    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:43.347137 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:43.347053    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:43.356794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:43.356769    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"]
Sep 11 16:55:43.358052 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:55:43.358024    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c4f14aa_2055_4c78_9549_0d905baa5723.slice/crio-bba04fb9fb284d81a353844eea3750650db6b35822885744160b249a72cb7960 WatchSource:0}: Error finding container bba04fb9fb284d81a353844eea3750650db6b35822885744160b249a72cb7960: Status 404 returned error can't find the container with id bba04fb9fb284d81a353844eea3750650db6b35822885744160b249a72cb7960
Sep 11 16:55:43.360487 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:43.360454    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:43.360563 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:43.360525    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:43.360563 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:43.360555    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:43.784762 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:43.784724    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" event={"ID":"3c4f14aa-2055-4c78-9549-0d905baa5723","Type":"ContainerStarted","Data":"b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc"}
Sep 11 16:55:43.785279 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:43.784787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" event={"ID":"3c4f14aa-2055-4c78-9549-0d905baa5723","Type":"ContainerStarted","Data":"bba04fb9fb284d81a353844eea3750650db6b35822885744160b249a72cb7960"}
Sep 11 16:55:43.819179 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:43.819123    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" podStartSLOduration=1.8191022989999999 podStartE2EDuration="1.819102299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:55:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:55:43.816965848 +0000 UTC m=+740.224184522" watchObservedRunningTime="2026-09-11 16:55:43.819102299 +0000 UTC m=+740.226320949"
Sep 11 16:55:43.938433 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:43.938396    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:44.068719 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:44.068604    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:44.076387 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:44.076353    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:44.106525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:44.106490    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:44.796986 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:44.796958    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:44.805700 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:44.805663    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:55:44.975720 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:44.975676    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:45.567799 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:45.567768    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:45.698090 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:45.697966    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:46.327102 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:46.327031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:47.484446 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:47.484394    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:48.321924 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:48.321865    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:48.420972 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:48.420936    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:48.524947 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:48.524914    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:48.791529 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:48.791497    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:48.876314 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:48.876280    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:49.408902 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:49.408858    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:49.501448 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:49.501423    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:49.634708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:49.634672    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:49.862185 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:49.862066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:50.028279 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.028250    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:50.170769 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.170742    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:50.325799 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.325761    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"]
Sep 11 16:55:50.329908 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.329885    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.332457 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.332435    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-32d680-openshift-default-dockercfg-vrlwp\""
Sep 11 16:55:50.345853 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.345825    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"]
Sep 11 16:55:50.354748 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.354721    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:50.383633 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.383593    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-credential-socket\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.383633 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.383638    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istiod-ca-cert\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.383820 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.383691    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-token\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.383820 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.383804    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-podinfo\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.383920 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.383907    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-data\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.383966 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.383952    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-envoy\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.384012 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.383978    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjr48\" (UniqueName: \"kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-kube-api-access-pjr48\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.384056 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.384025    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-certs\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.384107 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.384055    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-socket\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.464063 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.463977    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:50.484907 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.484874    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-socket\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.485115 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.484931    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-credential-socket\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.485115 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.484950    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istiod-ca-cert\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.485115 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.484974    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-token\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.485115 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.485031    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-podinfo\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.485115 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.485112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-data\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.485403 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.485148    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-envoy\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.485403 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.485169    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjr48\" (UniqueName: \"kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-kube-api-access-pjr48\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.485403 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.485198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-certs\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.485403 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.485382    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-socket\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.485607 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.485467    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-certs\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.485673 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.485658    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-data\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.486346 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.486299    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-credential-socket\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.486497 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.486473    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istiod-ca-cert\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.487949 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.487923    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-envoy\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.488036 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.487974    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-podinfo\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.513471 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.513436    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-token\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.513667 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.513506    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjr48\" (UniqueName: \"kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-kube-api-access-pjr48\") pod \"e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") " pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.588433 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.588394    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:50.641939 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.641893    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:50.706436 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.705882    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:50.791530 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.791433    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:50.911822 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.911660    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:50.935832 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.935776    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"]
Sep 11 16:55:50.945162 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:55:50.945133    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb65417_e286_4a09_a8ed_e2f20f0bf78f.slice/crio-1914d56addc1dd6acb145ad4a5df36e81618f6d11be968644ba11a7e5434c69f WatchSource:0}: Error finding container 1914d56addc1dd6acb145ad4a5df36e81618f6d11be968644ba11a7e5434c69f: Status 404 returned error can't find the container with id 1914d56addc1dd6acb145ad4a5df36e81618f6d11be968644ba11a7e5434c69f
Sep 11 16:55:50.948355 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.948242    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:50.948355 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.948339    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:50.948533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:50.948390    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:51.656175 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:51.656130    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:51.854607 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:51.854561    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" event={"ID":"5cb65417-e286-4a09-a8ed-e2f20f0bf78f","Type":"ContainerStarted","Data":"e12b80de64597e756bc76a0c8ec01255aeb2c5fd2a74d49c0053e13ff3507e9b"}
Sep 11 16:55:51.854798 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:51.854666    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" event={"ID":"5cb65417-e286-4a09-a8ed-e2f20f0bf78f","Type":"ContainerStarted","Data":"1914d56addc1dd6acb145ad4a5df36e81618f6d11be968644ba11a7e5434c69f"}
Sep 11 16:55:52.391553 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:52.391522    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:52.642574 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:52.642478    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:52.650343 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:52.650240    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:52.673892 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:52.673828    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" podStartSLOduration=2.673805172 podStartE2EDuration="2.673805172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:55:51.882391526 +0000 UTC m=+748.289610176" watchObservedRunningTime="2026-09-11 16:55:52.673805172 +0000 UTC m=+749.081023824"
Sep 11 16:55:52.862238 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:52.862192    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:52.863552 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:52.863524    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:55:53.188832 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:53.188792    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:55:53.505864 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:53.505757    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:53.603700 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:53.603670    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:54.875618 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:53.866915    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:54.875618 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:53.977494    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:54.875618 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:54.068236    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:54.875618 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:54.680318    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:54.875618 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:54.796746    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:54.889850 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:54.889751    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:54.898450 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:54.894016    2569 generic.go:356] "Generic (PLEG): container finished" podID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerID="703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15" exitCode=0
Sep 11 16:55:54.898450 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:54.894841    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" event={"ID":"d3bf67c2-4778-4df3-acc5-e64b3282c37f","Type":"ContainerDied","Data":"703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15"}
Sep 11 16:55:54.898450 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:54.895238    2569 scope.go:117] "RemoveContainer" containerID="703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15"
Sep 11 16:55:54.898450 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:54.896955    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:54.898450 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:54.897049    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:54.898450 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:54.897111    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:55.200478 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.200437    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:55.303216 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.303140    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:55.406871 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.406826    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:55.502797 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.502694    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:55.682025 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.681983    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:55.901981 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.901949    2569 generic.go:356] "Generic (PLEG): container finished" podID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerID="0cce37555d791bf6e2e5f44c3c471583712cf0750b30137419c3f2c76c72a4c5" exitCode=0
Sep 11 16:55:55.902462 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.901997    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" event={"ID":"1d3de30f-d85a-4e9d-94e6-35a490a3678b","Type":"ContainerDied","Data":"0cce37555d791bf6e2e5f44c3c471583712cf0750b30137419c3f2c76c72a4c5"}
Sep 11 16:55:55.902697 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.902672    2569 scope.go:117] "RemoveContainer" containerID="0cce37555d791bf6e2e5f44c3c471583712cf0750b30137419c3f2c76c72a4c5"
Sep 11 16:55:55.904443 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.904414    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:55.904539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.904499    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:55.904609 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.904543    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:55.905102 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.905023    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" event={"ID":"d3bf67c2-4778-4df3-acc5-e64b3282c37f","Type":"ContainerStarted","Data":"ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6"}
Sep 11 16:55:55.906922 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.906802    2569 generic.go:356] "Generic (PLEG): container finished" podID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerID="31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a" exitCode=0
Sep 11 16:55:55.906922 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.906844    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" event={"ID":"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91","Type":"ContainerDied","Data":"31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a"}
Sep 11 16:55:55.907324 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.907305    2569 scope.go:117] "RemoveContainer" containerID="31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a"
Sep 11 16:55:55.912401 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.912373    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:55.912577 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.912560    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:55.912704 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:55.912687    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:55:56.640838 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:56.640801    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:55:56.641050 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:56.640928    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 11 16:55:56.641050 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:56.640972    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 11 16:55:56.790579 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:56.790534    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:56.905597 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:56.905509    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:56.942672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:56.941593    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" event={"ID":"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91","Type":"ContainerStarted","Data":"d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f"}
Sep 11 16:55:56.950419 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:56.950149    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" event={"ID":"1d3de30f-d85a-4e9d-94e6-35a490a3678b","Type":"ContainerStarted","Data":"0e965230c431a11d402b2c4e1bb33e49439a4eee417687bd9254a99d9424952f"}
Sep 11 16:55:57.592424 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:57.592385    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:57.640646 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:57.640604    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 11 16:55:57.640843 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:57.640669    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 11 16:55:57.777846 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:57.777806    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:55:57.778114 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:57.778059    2569 patch_prober.go:28] interesting pod/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 11 16:55:57.778214 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:57.778146    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 11 16:55:57.897326 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:57.897282    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:55:57.897548 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:57.897424    2569 patch_prober.go:28] interesting pod/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 16:55:57.897548 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:57.897474    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 16:55:58.555098 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:58.552978    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:58.641122 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:58.640484    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 11 16:55:58.641122 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:58.640553    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 11 16:55:58.653100 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:58.650798    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:58.750042 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:58.749999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:58.776598 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:58.776555    2569 patch_prober.go:28] interesting pod/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 11 16:55:58.776807 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:58.776637    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 11 16:55:58.897213 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:58.897169    2569 patch_prober.go:28] interesting pod/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 16:55:58.897418 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:58.897267    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 16:55:59.089642 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:59.089578    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:59.187832 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:59.187605    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:59.591711 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:59.591604    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:55:59.640548 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:59.640496    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 11 16:55:59.640737 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:59.640572    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 11 16:55:59.779570 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:59.779529    2569 patch_prober.go:28] interesting pod/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 11 16:55:59.779895 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:59.779862    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 11 16:55:59.897001 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:59.896962    2569 patch_prober.go:28] interesting pod/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 16:55:59.897359 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:55:59.897329    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 16:56:00.372216 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:00.372110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:00.640182 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:00.640141    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 11 16:56:00.640817 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:00.640780    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 11 16:56:00.776606 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:00.776560    2569 patch_prober.go:28] interesting pod/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 11 16:56:00.776805 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:00.776634    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 11 16:56:00.897161 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:00.896917    2569 patch_prober.go:28] interesting pod/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 16:56:00.897161 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:00.896985    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 16:56:01.339418 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:01.339319    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:01.640190 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:01.640149    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 11 16:56:01.640785 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:01.640751    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 11 16:56:01.776980 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:01.776946    2569 patch_prober.go:28] interesting pod/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 11 16:56:01.777336 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:01.777281    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 11 16:56:01.924170 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:01.896554    2569 patch_prober.go:28] interesting pod/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 16:56:01.924170 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:01.896623    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 16:56:02.277043 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:02.276887    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:02.644625 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:02.644580    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 11 16:56:02.645165 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:02.644648    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 11 16:56:02.777436 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:02.777386    2569 patch_prober.go:28] interesting pod/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 11 16:56:02.777649 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:02.777460    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 11 16:56:02.897060 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:02.896937    2569 patch_prober.go:28] interesting pod/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 16:56:02.897060 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:02.897009    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 16:56:03.015545 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:03.011439    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:03.640778 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:03.640731    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 11 16:56:03.640992 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:03.640806    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 11 16:56:03.728711 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:03.728672    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:56:03.776981 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:03.776824    2569 patch_prober.go:28] interesting pod/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 11 16:56:03.776981 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:03.776900    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 11 16:56:03.896750 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:03.896637    2569 patch_prober.go:28] interesting pod/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 16:56:03.896750 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:03.896715    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 16:56:03.928414 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:03.928354    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:04.030828 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:04.030624    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:04.646301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:04.646266    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:56:04.647498 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:04.647474    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 16:56:04.733793 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:04.732841    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:56:04.734397 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:04.734359    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:56:04.782459 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:04.782426    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:56:04.902136 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:04.902054    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:56:05.031991 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:05.031963    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 16:56:05.032208 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:05.032065    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 16:56:05.242518 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:05.242438    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:05.799245 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:05.799206    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:06.662656 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:06.662599    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:07.059954 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.059852    2569 generic.go:356] "Generic (PLEG): container finished" podID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerID="b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc" exitCode=0
Sep 11 16:56:07.060449 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.059949    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" event={"ID":"3c4f14aa-2055-4c78-9549-0d905baa5723","Type":"ContainerDied","Data":"b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc"}
Sep 11 16:56:07.060542 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.060522    2569 scope.go:117] "RemoveContainer" containerID="b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc"
Sep 11 16:56:07.061771 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.061754    2569 generic.go:356] "Generic (PLEG): container finished" podID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerID="283931ece6996db539885cdc09f0d7ef887983e61b9f33d107a924f46e0382d0" exitCode=0
Sep 11 16:56:07.061837 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.061782    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" event={"ID":"15380047-35b2-4d02-8bb4-81a0c09a6519","Type":"ContainerDied","Data":"283931ece6996db539885cdc09f0d7ef887983e61b9f33d107a924f46e0382d0"}
Sep 11 16:56:07.061892 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.061870    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:07.061962 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.061950    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:07.061998 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.061983    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:07.062165 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.062151    2569 scope.go:117] "RemoveContainer" containerID="283931ece6996db539885cdc09f0d7ef887983e61b9f33d107a924f46e0382d0"
Sep 11 16:56:07.063070 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.063047    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:07.063160 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.063140    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:07.063202 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.063174    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:07.477269 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:07.477232    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:08.074526 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:08.074480    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" event={"ID":"15380047-35b2-4d02-8bb4-81a0c09a6519","Type":"ContainerStarted","Data":"cfc63baf6e8c7667b9f756a70fd7553f527a01aa0faf556cf45942c7470a2930"}
Sep 11 16:56:08.077881 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:08.077845    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" event={"ID":"3c4f14aa-2055-4c78-9549-0d905baa5723","Type":"ContainerStarted","Data":"03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb"}
Sep 11 16:56:08.575746 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:08.575698    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:08.630803 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:08.630762    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:56:08.631859 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:08.631817    2569 patch_prober.go:28] interesting pod/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 11 16:56:08.632018 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:08.631880    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 11 16:56:09.068685 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:09.068645    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:56:09.069156 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:09.069130    2569 patch_prober.go:28] interesting pod/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 16:56:09.069396 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:09.069370    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 16:56:09.375269 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:09.375156    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:09.472353 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:09.472294    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:09.578480 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:09.578417    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:09.630805 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:09.630769    2569 patch_prober.go:28] interesting pod/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 11 16:56:09.631122 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:09.631065    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 11 16:56:09.680418 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:09.680374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:10.068724 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:10.068620    2569 patch_prober.go:28] interesting pod/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 16:56:10.068991 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:10.068956    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 16:56:10.505207 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:10.505173    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:10.630511 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:10.630470    2569 patch_prober.go:28] interesting pod/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 11 16:56:10.630786 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:10.630757    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 11 16:56:11.057240 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.057196    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"]
Sep 11 16:56:11.061086 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.061035    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"
Sep 11 16:56:11.063586 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.063559    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:56:11.063851 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.063633    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 16:56:11.068414 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.068382    2569 patch_prober.go:28] interesting pod/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 16:56:11.068679 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.068651    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 16:56:11.076632 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.076595    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"]
Sep 11 16:56:11.159818 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.159766    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c32e8cd3-3a29-4f3c-bf09-11511c7d931d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k\" (UID: \"c32e8cd3-3a29-4f3c-bf09-11511c7d931d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"
Sep 11 16:56:11.266100 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.262151    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c32e8cd3-3a29-4f3c-bf09-11511c7d931d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k\" (UID: \"c32e8cd3-3a29-4f3c-bf09-11511c7d931d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"
Sep 11 16:56:11.266100 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.263554    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c32e8cd3-3a29-4f3c-bf09-11511c7d931d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k\" (UID: \"c32e8cd3-3a29-4f3c-bf09-11511c7d931d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"
Sep 11 16:56:11.385104 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.378157    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"
Sep 11 16:56:11.508858 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.508798    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:11.613874 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.613835    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:56:11.631585 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.631539    2569 patch_prober.go:28] interesting pod/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 11 16:56:11.631791 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.631609    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 11 16:56:11.703898 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.703627    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"]
Sep 11 16:56:11.708122 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:56:11.708085    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc32e8cd3_3a29_4f3c_bf09_11511c7d931d.slice/crio-28feb23c38465cf0b4b28b1809ccc6e0695e20b451ba0376323d1a1f871e62c6 WatchSource:0}: Error finding container 28feb23c38465cf0b4b28b1809ccc6e0695e20b451ba0376323d1a1f871e62c6: Status 404 returned error can't find the container with id 28feb23c38465cf0b4b28b1809ccc6e0695e20b451ba0376323d1a1f871e62c6
Sep 11 16:56:11.710822 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:11.710800    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 16:56:12.068477 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:12.068365    2569 patch_prober.go:28] interesting pod/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 16:56:12.068477 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:12.068441    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 16:56:12.138833 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:12.138785    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k" event={"ID":"c32e8cd3-3a29-4f3c-bf09-11511c7d931d","Type":"ContainerStarted","Data":"28feb23c38465cf0b4b28b1809ccc6e0695e20b451ba0376323d1a1f871e62c6"}
Sep 11 16:56:12.374146 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:12.373967    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:12.641058 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:12.640593    2569 patch_prober.go:28] interesting pod/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 11 16:56:12.641058 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:12.640660    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 11 16:56:12.777152 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:12.776185    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:56:13.070049 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:13.069525    2569 patch_prober.go:28] interesting pod/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 16:56:13.070049 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:13.069593    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 16:56:13.123231 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:13.123191    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:13.637109 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:13.637039    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:56:13.638622 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:13.638558    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:56:13.796190 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:13.796151    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:13.953592 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:13.953510    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:14.074370 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:14.074335    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:56:14.075011 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:14.074746    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:56:14.075557 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:14.075536    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:56:14.788665 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:14.788582    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:15.173069 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:15.173031    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k" event={"ID":"c32e8cd3-3a29-4f3c-bf09-11511c7d931d","Type":"ContainerStarted","Data":"ce4a61b39e0421bb46b2a35e3fd119c11b20b12d29a1af697a902790cf9c952a"}
Sep 11 16:56:15.193823 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:15.193760    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k" podStartSLOduration=1.366795858 podStartE2EDuration="4.193741388s" totalImagesPullingTime="2.82694553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:56:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:56:15.193335843 +0000 UTC m=+771.600554511" watchObservedRunningTime="2026-09-11 16:56:15.193741388 +0000 UTC m=+771.600960035"
Sep 11 16:56:15.683449 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:15.683366    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:16.179213 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:16.179179    2569 generic.go:356] "Generic (PLEG): container finished" podID="c32e8cd3-3a29-4f3c-bf09-11511c7d931d" containerID="ce4a61b39e0421bb46b2a35e3fd119c11b20b12d29a1af697a902790cf9c952a" exitCode=0
Sep 11 16:56:16.179620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:16.179254    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k" event={"ID":"c32e8cd3-3a29-4f3c-bf09-11511c7d931d","Type":"ContainerDied","Data":"ce4a61b39e0421bb46b2a35e3fd119c11b20b12d29a1af697a902790cf9c952a"}
Sep 11 16:56:16.180850 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:16.180831    2569 generic.go:356] "Generic (PLEG): container finished" podID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerID="e12b80de64597e756bc76a0c8ec01255aeb2c5fd2a74d49c0053e13ff3507e9b" exitCode=0
Sep 11 16:56:16.180967 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:16.180906    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" event={"ID":"5cb65417-e286-4a09-a8ed-e2f20f0bf78f","Type":"ContainerDied","Data":"e12b80de64597e756bc76a0c8ec01255aeb2c5fd2a74d49c0053e13ff3507e9b"}
Sep 11 16:56:16.181352 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:16.181337    2569 scope.go:117] "RemoveContainer" containerID="e12b80de64597e756bc76a0c8ec01255aeb2c5fd2a74d49c0053e13ff3507e9b"
Sep 11 16:56:16.182559 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:16.182534    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:16.182619 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:16.182596    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:16.182658 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:16.182647    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:16.467214 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:16.467122    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:16.798147 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:16.798043    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:16.898791 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:16.898755    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:17.002457 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:17.002422    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:17.188165 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:17.188119    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" event={"ID":"5cb65417-e286-4a09-a8ed-e2f20f0bf78f","Type":"ContainerStarted","Data":"89aca449932f3dcd41515e5b375a39fe931f2dbcb8640763d8f6154de39b7b35"}
Sep 11 16:56:17.226737 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:17.226706    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:17.362796 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:17.362771    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"
Sep 11 16:56:17.395143 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:17.395016    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:17.444399 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:17.443550    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c32e8cd3-3a29-4f3c-bf09-11511c7d931d-maas-api-service-ca\") pod \"c32e8cd3-3a29-4f3c-bf09-11511c7d931d\" (UID: \"c32e8cd3-3a29-4f3c-bf09-11511c7d931d\") "
Sep 11 16:56:17.444399 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:17.444293    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c32e8cd3-3a29-4f3c-bf09-11511c7d931d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c32e8cd3-3a29-4f3c-bf09-11511c7d931d" (UID: "c32e8cd3-3a29-4f3c-bf09-11511c7d931d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:56:17.544892 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:17.544852    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c32e8cd3-3a29-4f3c-bf09-11511c7d931d-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:56:17.643158 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:17.643121    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:56:17.643362 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:17.643245    2569 patch_prober.go:28] interesting pod/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 11 16:56:17.643362 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:17.643296    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 11 16:56:18.102049 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:18.102017    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:18.194334 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:18.194306    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"
Sep 11 16:56:18.194718 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:18.194351    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k" event={"ID":"c32e8cd3-3a29-4f3c-bf09-11511c7d931d","Type":"ContainerDied","Data":"28feb23c38465cf0b4b28b1809ccc6e0695e20b451ba0376323d1a1f871e62c6"}
Sep 11 16:56:18.194718 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:18.194399    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28feb23c38465cf0b4b28b1809ccc6e0695e20b451ba0376323d1a1f871e62c6"
Sep 11 16:56:18.642526 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:18.642484    2569 patch_prober.go:28] interesting pod/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 11 16:56:18.642829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:18.642794    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 11 16:56:18.884344 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:18.884308    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:19.642921 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:19.642872    2569 patch_prober.go:28] interesting pod/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 11 16:56:19.643480 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:19.642946    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 11 16:56:19.719342 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:19.719307    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:20.642720 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:20.642674    2569 patch_prober.go:28] interesting pod/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 11 16:56:20.642920 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:20.642746    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 11 16:56:21.642990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:21.642525    2569 patch_prober.go:28] interesting pod/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 11 16:56:21.642990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:21.642590    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 11 16:56:22.646163 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:22.646134    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:56:22.646580 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:22.646415    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:56:22.647060 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:22.647043    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 16:56:24.333156 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:24.333125    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:24.467417 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:24.467382    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:25.764091 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.764048    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"]
Sep 11 16:56:25.764500 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.764483    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c32e8cd3-3a29-4f3c-bf09-11511c7d931d" containerName="revoke-keys"
Sep 11 16:56:25.764500 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.764499    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c32e8cd3-3a29-4f3c-bf09-11511c7d931d" containerName="revoke-keys"
Sep 11 16:56:25.764618 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.764572    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c32e8cd3-3a29-4f3c-bf09-11511c7d931d" containerName="revoke-keys"
Sep 11 16:56:25.772210 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.772186    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.774448 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.774428    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-52b0af\"/\"kube-root-ca.crt\""
Sep 11 16:56:25.774943 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.774918    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-52b0af\"/\"default-dockercfg-c2dgz\""
Sep 11 16:56:25.775039 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.774951    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-52b0af\"/\"openshift-service-ca.crt\""
Sep 11 16:56:25.775039 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.774965    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-52b0af\"/\"simulator-w0-52b0af-kserve-self-signed-certs\""
Sep 11 16:56:25.778477 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.778457    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"]
Sep 11 16:56:25.811332 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.811282    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:25.834796 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.834771    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-home\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.834911 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.834808    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq9fv\" (UniqueName: \"kubernetes.io/projected/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-kube-api-access-rq9fv\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.834981 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.834905    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-dshm\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.834981 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.834949    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tls-certs\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.835124 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.835015    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-model-cache\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.835189 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.835125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tmp-dir\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.935656 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.935621    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-home\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.935859 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.935666    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq9fv\" (UniqueName: \"kubernetes.io/projected/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-kube-api-access-rq9fv\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.935859 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.935713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-dshm\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.935859 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.935737    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tls-certs\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.935859 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.935768    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-model-cache\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.935859 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.935824    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tmp-dir\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.936206 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.936171    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-home\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.936251 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.936209    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-model-cache\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.936251 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.936235    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tmp-dir\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.938107 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.938067    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-dshm\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.938495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.938475    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tls-certs\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:25.943927 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:25.943906    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq9fv\" (UniqueName: \"kubernetes.io/projected/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-kube-api-access-rq9fv\") pod \"simulator-w0-52b0af-kserve-668d7b6699-g5pn9\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") " pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:26.081940 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:26.081843    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:26.361454 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:26.361428    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"]
Sep 11 16:56:26.363477 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:56:26.363449    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87c1aebf_e5dd_4a88_a827_e79ee7beadf3.slice/crio-3256ae18e19aa3e969e3fb5a20c74e3ba1bb37708b78bd96166df1e04160adb0 WatchSource:0}: Error finding container 3256ae18e19aa3e969e3fb5a20c74e3ba1bb37708b78bd96166df1e04160adb0: Status 404 returned error can't find the container with id 3256ae18e19aa3e969e3fb5a20c74e3ba1bb37708b78bd96166df1e04160adb0
Sep 11 16:56:27.246889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:27.246859    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9" event={"ID":"87c1aebf-e5dd-4a88-a827-e79ee7beadf3","Type":"ContainerStarted","Data":"b03a7305a24b08291c20944012974f7e984da4467a56b8e36de0ebd1d3d114f2"}
Sep 11 16:56:27.247350 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:27.246904    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9" event={"ID":"87c1aebf-e5dd-4a88-a827-e79ee7beadf3","Type":"ContainerStarted","Data":"3256ae18e19aa3e969e3fb5a20c74e3ba1bb37708b78bd96166df1e04160adb0"}
Sep 11 16:56:27.265096 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:27.264872    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:27.266472 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:27.266411    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9" podStartSLOduration=2.26639065 podStartE2EDuration="2.26639065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:56:27.265187713 +0000 UTC m=+783.672406408" watchObservedRunningTime="2026-09-11 16:56:27.26639065 +0000 UTC m=+783.673609302"
Sep 11 16:56:27.956491 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:27.956447    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:28.045879 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:28.045832    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:28.117098 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:28.112918    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"]
Sep 11 16:56:28.121308 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:28.121275    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab1ade68-4zf5k"]
Sep 11 16:56:28.152007 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:28.151972    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:28.209868 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:28.209767    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c32e8cd3-3a29-4f3c-bf09-11511c7d931d" path="/var/lib/kubelet/pods/c32e8cd3-3a29-4f3c-bf09-11511c7d931d/volumes"
Sep 11 16:56:28.331121 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:28.331065    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"]
Sep 11 16:56:28.331506 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:28.331358    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy" containerID="cri-o://03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb" gracePeriod=30
Sep 11 16:56:28.371749 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:28.371711    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:28.666424 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:28.666383    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:28.908866 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:28.908831    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:29.052765 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.052684    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:29.078271 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.078230    2569 patch_prober.go:28] interesting pod/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 16:56:29.078457 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.078302    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 16:56:29.167541 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.167504    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:29.430626 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.430592    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"]
Sep 11 16:56:29.434818 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.434795    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.438363 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.438339    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-5969a028-gw-openshift-default-dockercfg-7h8c4\""
Sep 11 16:56:29.448372 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.448336    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"]
Sep 11 16:56:29.493271 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.493238    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:29.572641 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.572605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-envoy\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.572845 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.572685    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-certs\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.572845 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.572716    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istiod-ca-cert\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.572845 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.572753    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-credential-socket\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.572976 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.572857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv8n2\" (UniqueName: \"kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-kube-api-access-lv8n2\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.572976 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.572918    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-token\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.572976 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.572956    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-socket\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.573117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.572990    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-podinfo\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.573117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.573034    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-data\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.668591 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.668561    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:29.673881 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.673845    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv8n2\" (UniqueName: \"kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-kube-api-access-lv8n2\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.673993 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.673900    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-token\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.673993 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.673927    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-socket\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.673993 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.673950    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-podinfo\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.673993 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.673976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-data\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.674210 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.674020    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-envoy\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.674210 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.674118    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-certs\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.674210 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.674151    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istiod-ca-cert\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.674343 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.674210    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-credential-socket\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.674419 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.674395    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-socket\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.674513 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.674477    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-data\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.674608 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.674549    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-certs\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.674659 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.674610    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-credential-socket\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.674916 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.674897    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istiod-ca-cert\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.676495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.676464    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-envoy\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.676620 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.676600    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-podinfo\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.682596 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.682539    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-token\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.682967 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.682948    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv8n2\" (UniqueName: \"kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-kube-api-access-lv8n2\") pod \"e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") " pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:29.746375 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:29.746328    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:30.028761 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:30.028730    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"]
Sep 11 16:56:30.030969 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:56:30.030942    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4eaecf_2c80_4660_8d05_9d6a4c24d6d1.slice/crio-7d18a175f9556c553a7c006f3a553000668ab59af204a9620375d37e8424228e WatchSource:0}: Error finding container 7d18a175f9556c553a7c006f3a553000668ab59af204a9620375d37e8424228e: Status 404 returned error can't find the container with id 7d18a175f9556c553a7c006f3a553000668ab59af204a9620375d37e8424228e
Sep 11 16:56:30.033489 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:30.033453    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:30.033565 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:30.033550    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:30.033610 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:30.033593    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:30.223216 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:30.223179    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:30.269911 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:30.269867    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" event={"ID":"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1","Type":"ContainerStarted","Data":"ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c"}
Sep 11 16:56:30.270133 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:30.269941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" event={"ID":"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1","Type":"ContainerStarted","Data":"7d18a175f9556c553a7c006f3a553000668ab59af204a9620375d37e8424228e"}
Sep 11 16:56:30.291866 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:30.291685    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" podStartSLOduration=1.291637212 podStartE2EDuration="1.291637212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:56:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:56:30.291174999 +0000 UTC m=+786.698393672" watchObservedRunningTime="2026-09-11 16:56:30.291637212 +0000 UTC m=+786.698855862"
Sep 11 16:56:30.563657 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:30.563552    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:30.747288 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:30.747246    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:31.374815 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:31.374781    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:31.752006 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:31.751972    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:31.806097 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:31.803020    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:31.885010 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:31.884976    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:32.281404 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:32.281373    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:32.282294 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:32.282273    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:32.317860 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:32.317825    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:56:32.470711 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:32.470678    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:33.614999 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.614973    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:56:33.718043 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.717948    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-credential-socket\") pod \"3c4f14aa-2055-4c78-9549-0d905baa5723\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") "
Sep 11 16:56:33.718043 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718014    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-data\") pod \"3c4f14aa-2055-4c78-9549-0d905baa5723\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") "
Sep 11 16:56:33.718321 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718053    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-socket\") pod \"3c4f14aa-2055-4c78-9549-0d905baa5723\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") "
Sep 11 16:56:33.718321 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718167    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-certs\") pod \"3c4f14aa-2055-4c78-9549-0d905baa5723\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") "
Sep 11 16:56:33.718321 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718234    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4f14aa-2055-4c78-9549-0d905baa5723-istiod-ca-cert\") pod \"3c4f14aa-2055-4c78-9549-0d905baa5723\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") "
Sep 11 16:56:33.718321 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718260    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpsw9\" (UniqueName: \"kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-kube-api-access-mpsw9\") pod \"3c4f14aa-2055-4c78-9549-0d905baa5723\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") "
Sep 11 16:56:33.718533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718336    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c4f14aa-2055-4c78-9549-0d905baa5723" (UID: "3c4f14aa-2055-4c78-9549-0d905baa5723"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:33.718533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718338    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c4f14aa-2055-4c78-9549-0d905baa5723" (UID: "3c4f14aa-2055-4c78-9549-0d905baa5723"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:33.718533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718342    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-token\") pod \"3c4f14aa-2055-4c78-9549-0d905baa5723\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") "
Sep 11 16:56:33.718533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718348    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c4f14aa-2055-4c78-9549-0d905baa5723" (UID: "3c4f14aa-2055-4c78-9549-0d905baa5723"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:33.718533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718408    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c4f14aa-2055-4c78-9549-0d905baa5723" (UID: "3c4f14aa-2055-4c78-9549-0d905baa5723"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:33.718533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718456    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-podinfo\") pod \"3c4f14aa-2055-4c78-9549-0d905baa5723\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") "
Sep 11 16:56:33.718533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718491    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-envoy\") pod \"3c4f14aa-2055-4c78-9549-0d905baa5723\" (UID: \"3c4f14aa-2055-4c78-9549-0d905baa5723\") "
Sep 11 16:56:33.718889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718656    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c4f14aa-2055-4c78-9549-0d905baa5723-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c4f14aa-2055-4c78-9549-0d905baa5723" (UID: "3c4f14aa-2055-4c78-9549-0d905baa5723"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:56:33.719005 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718927    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:56:33.719005 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718949    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:56:33.719005 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718963    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:56:33.719005 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718976    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4f14aa-2055-4c78-9549-0d905baa5723-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:56:33.719005 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.718989    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:56:33.720825 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.720791    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c4f14aa-2055-4c78-9549-0d905baa5723" (UID: "3c4f14aa-2055-4c78-9549-0d905baa5723"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:56:33.720825 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.720803    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c4f14aa-2055-4c78-9549-0d905baa5723" (UID: "3c4f14aa-2055-4c78-9549-0d905baa5723"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:33.721004 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.720984    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c4f14aa-2055-4c78-9549-0d905baa5723" (UID: "3c4f14aa-2055-4c78-9549-0d905baa5723"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:56:33.721004 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.720996    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-kube-api-access-mpsw9" (OuterVolumeSpecName: "kube-api-access-mpsw9") pod "3c4f14aa-2055-4c78-9549-0d905baa5723" (UID: "3c4f14aa-2055-4c78-9549-0d905baa5723"). InnerVolumeSpecName "kube-api-access-mpsw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:56:33.771522 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.771474    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:33.820021 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.819975    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpsw9\" (UniqueName: \"kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-kube-api-access-mpsw9\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:56:33.820021 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.820008    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:56:33.820021 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.820022    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:56:33.820021 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:33.820030    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4f14aa-2055-4c78-9549-0d905baa5723-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:56:34.293026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.292974    2569 generic.go:356] "Generic (PLEG): container finished" podID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerID="03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb" exitCode=0
Sep 11 16:56:34.293026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.293025    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"
Sep 11 16:56:34.293276 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.293042    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" event={"ID":"3c4f14aa-2055-4c78-9549-0d905baa5723","Type":"ContainerDied","Data":"03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb"}
Sep 11 16:56:34.293276 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.293119    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj" event={"ID":"3c4f14aa-2055-4c78-9549-0d905baa5723","Type":"ContainerDied","Data":"bba04fb9fb284d81a353844eea3750650db6b35822885744160b249a72cb7960"}
Sep 11 16:56:34.293276 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.293151    2569 scope.go:117] "RemoveContainer" containerID="03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb"
Sep 11 16:56:34.302687 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.302667    2569 scope.go:117] "RemoveContainer" containerID="b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc"
Sep 11 16:56:34.312162 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.312096    2569 scope.go:117] "RemoveContainer" containerID="03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb"
Sep 11 16:56:34.312543 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:56:34.312518    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb\": container with ID starting with 03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb not found: ID does not exist" containerID="03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb"
Sep 11 16:56:34.312670 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.312550    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb"} err="failed to get container status \"03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb\": rpc error: code = NotFound desc = could not find container \"03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb\": container with ID starting with 03423485b10a5f843e472ec0c4804a148ceeb88d62d3e4388940091d2a45bebb not found: ID does not exist"
Sep 11 16:56:34.312670 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.312574    2569 scope.go:117] "RemoveContainer" containerID="b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc"
Sep 11 16:56:34.312928 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:56:34.312901    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc\": container with ID starting with b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc not found: ID does not exist" containerID="b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc"
Sep 11 16:56:34.313047 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.312930    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc"} err="failed to get container status \"b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc\": rpc error: code = NotFound desc = could not find container \"b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc\": container with ID starting with b86aaf99350b428ab54f07dc0f741091cea0ff7eb30abfbe99af98527b6edfbc not found: ID does not exist"
Sep 11 16:56:34.314195 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.314170    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"]
Sep 11 16:56:34.317469 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.317445    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ab1ade68-openshift-default-bb68fd844-klxqj"]
Sep 11 16:56:34.509931 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.509851    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:34.614234 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.614198    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:34.938125 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:34.938069    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:35.055202 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:35.055169    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:35.155146 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:35.155070    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:35.474276 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:35.474239    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:36.082430 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:36.082392    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:36.082430 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:36.082435    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:36.095102 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:36.095060    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:36.205906 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:36.205871    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" path="/var/lib/kubelet/pods/3c4f14aa-2055-4c78-9549-0d905baa5723/volumes"
Sep 11 16:56:36.339381 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:36.339302    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 16:56:36.487656 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:36.487622    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:37.203539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:37.203503    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:38.075385 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:38.075350    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:42.342684 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:42.342653    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:53.416675 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:53.416638    2569 generic.go:356] "Generic (PLEG): container finished" podID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerID="ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c" exitCode=0
Sep 11 16:56:53.417242 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:53.416696    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" event={"ID":"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1","Type":"ContainerDied","Data":"ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c"}
Sep 11 16:56:53.417326 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:53.417240    2569 scope.go:117] "RemoveContainer" containerID="ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c"
Sep 11 16:56:53.418504 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:53.418471    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:53.418591 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:53.418543    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:53.418591 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:53.418573    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:56:53.693399 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:53.693310    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:56:54.424282 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:54.424250    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" event={"ID":"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1","Type":"ContainerStarted","Data":"11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec"}
Sep 11 16:56:54.746739 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:54.746623    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:54.746999 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:54.746971    2569 patch_prober.go:28] interesting pod/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 11 16:56:54.747095 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:54.747036    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 11 16:56:55.747761 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:55.747715    2569 patch_prober.go:28] interesting pod/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 11 16:56:55.748317 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:55.747797    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 11 16:56:56.729252 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.729214    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:56.747389 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.747342    2569 patch_prober.go:28] interesting pod/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 11 16:56:56.747597 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.747427    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 11 16:56:56.873905 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.873869    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"]
Sep 11 16:56:56.874359 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.874346    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy"
Sep 11 16:56:56.874359 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.874360    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy"
Sep 11 16:56:56.874434 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.874368    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy"
Sep 11 16:56:56.874434 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.874374    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy"
Sep 11 16:56:56.874497 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.874442    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy"
Sep 11 16:56:56.874497 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.874454    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c4f14aa-2055-4c78-9549-0d905baa5723" containerName="istio-proxy"
Sep 11 16:56:56.877318 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.877302    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"
Sep 11 16:56:56.879554 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.879529    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 16:56:56.879554 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.879548    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:56:56.886786 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.886755    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"]
Sep 11 16:56:56.970573 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:56.970538    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec59983-b446-4cf6-a8ba-2b28ddff84fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5969a028-4jks6\" (UID: \"8ec59983-b446-4cf6-a8ba-2b28ddff84fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"
Sep 11 16:56:57.072035 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:57.071943    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec59983-b446-4cf6-a8ba-2b28ddff84fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5969a028-4jks6\" (UID: \"8ec59983-b446-4cf6-a8ba-2b28ddff84fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"
Sep 11 16:56:57.072635 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:57.072615    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec59983-b446-4cf6-a8ba-2b28ddff84fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5969a028-4jks6\" (UID: \"8ec59983-b446-4cf6-a8ba-2b28ddff84fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"
Sep 11 16:56:57.188670 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:57.188634    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"
Sep 11 16:56:57.495957 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:57.495931    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"]
Sep 11 16:56:57.498176 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:56:57.498150    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec59983_b446_4cf6_a8ba_2b28ddff84fc.slice/crio-ad416eaab52a0fdde63384da442aee73c53aa07543ff9f5df459c0f96f5835fa WatchSource:0}: Error finding container ad416eaab52a0fdde63384da442aee73c53aa07543ff9f5df459c0f96f5835fa: Status 404 returned error can't find the container with id ad416eaab52a0fdde63384da442aee73c53aa07543ff9f5df459c0f96f5835fa
Sep 11 16:56:57.672353 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:57.672315    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:57.747854 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:57.747753    2569 patch_prober.go:28] interesting pod/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 11 16:56:57.748525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:57.748121    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 11 16:56:58.370711 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:58.370660    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:58.463028 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:58.461805    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6" event={"ID":"8ec59983-b446-4cf6-a8ba-2b28ddff84fc","Type":"ContainerStarted","Data":"088e4a3b08e5aaf497bc1587c3fe4e08441669bc9b5b3cd4eb815b3577125a2d"}
Sep 11 16:56:58.463028 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:58.461874    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6" event={"ID":"8ec59983-b446-4cf6-a8ba-2b28ddff84fc","Type":"ContainerStarted","Data":"ad416eaab52a0fdde63384da442aee73c53aa07543ff9f5df459c0f96f5835fa"}
Sep 11 16:56:58.478483 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:58.478433    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6" podStartSLOduration=2.478414159 podStartE2EDuration="2.478414159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:56:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:56:58.477836986 +0000 UTC m=+814.885055635" watchObservedRunningTime="2026-09-11 16:56:58.478414159 +0000 UTC m=+814.885632808"
Sep 11 16:56:58.747645 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:58.747203    2569 patch_prober.go:28] interesting pod/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 11 16:56:58.747645 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:58.747265    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 11 16:56:59.178834 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:59.178798    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:56:59.469755 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:59.469663    2569 generic.go:356] "Generic (PLEG): container finished" podID="8ec59983-b446-4cf6-a8ba-2b28ddff84fc" containerID="088e4a3b08e5aaf497bc1587c3fe4e08441669bc9b5b3cd4eb815b3577125a2d" exitCode=0
Sep 11 16:56:59.470128 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:59.469750    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6" event={"ID":"8ec59983-b446-4cf6-a8ba-2b28ddff84fc","Type":"ContainerDied","Data":"088e4a3b08e5aaf497bc1587c3fe4e08441669bc9b5b3cd4eb815b3577125a2d"}
Sep 11 16:56:59.780151 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:59.780046    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:59.780556 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:59.780531    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:56:59.781987 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:56:59.781964    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:57:00.011039 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:00.011002    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:00.611137 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:00.611115    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"
Sep 11 16:57:00.713001 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:00.712963    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec59983-b446-4cf6-a8ba-2b28ddff84fc-maas-api-service-ca\") pod \"8ec59983-b446-4cf6-a8ba-2b28ddff84fc\" (UID: \"8ec59983-b446-4cf6-a8ba-2b28ddff84fc\") "
Sep 11 16:57:00.713374 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:00.713352    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ec59983-b446-4cf6-a8ba-2b28ddff84fc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8ec59983-b446-4cf6-a8ba-2b28ddff84fc" (UID: "8ec59983-b446-4cf6-a8ba-2b28ddff84fc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:57:00.761119 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:00.761021    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:00.814712 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:00.814679    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec59983-b446-4cf6-a8ba-2b28ddff84fc-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:01.484006 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:01.483974    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"
Sep 11 16:57:01.484203 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:01.484025    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6" event={"ID":"8ec59983-b446-4cf6-a8ba-2b28ddff84fc","Type":"ContainerDied","Data":"ad416eaab52a0fdde63384da442aee73c53aa07543ff9f5df459c0f96f5835fa"}
Sep 11 16:57:01.484203 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:01.484070    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad416eaab52a0fdde63384da442aee73c53aa07543ff9f5df459c0f96f5835fa"
Sep 11 16:57:01.648522 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:01.648484    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:02.457183 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:02.457147    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:03.100696 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:03.100668    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:03.179738 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:03.179710    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:03.280182 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:03.280150    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:03.333815 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:03.333783    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"]
Sep 11 16:57:03.336804 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:03.336777    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5969a028-4jks6"]
Sep 11 16:57:03.354826 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:03.354754    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:03.494211 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:03.494180    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"]
Sep 11 16:57:03.494436 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:03.494401    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy" containerID="cri-o://11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec" gracePeriod=30
Sep 11 16:57:03.497485 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:03.497446    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:03.836192 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:03.836158    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:03.972068 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:03.972031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:04.206172 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.206138    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ec59983-b446-4cf6-a8ba-2b28ddff84fc" path="/var/lib/kubelet/pods/8ec59983-b446-4cf6-a8ba-2b28ddff84fc/volumes"
Sep 11 16:57:04.279349 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.279315    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"]
Sep 11 16:57:04.279779 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.279764    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec59983-b446-4cf6-a8ba-2b28ddff84fc" containerName="revoke-keys"
Sep 11 16:57:04.279779 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.279780    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec59983-b446-4cf6-a8ba-2b28ddff84fc" containerName="revoke-keys"
Sep 11 16:57:04.279911 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.279851    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ec59983-b446-4cf6-a8ba-2b28ddff84fc" containerName="revoke-keys"
Sep 11 16:57:04.283301 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.283277    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.285456 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.285434    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d310aa61-openshift-default-dockercfg-spr4p\""
Sep 11 16:57:04.295713 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.295690    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"]
Sep 11 16:57:04.312001 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.311959    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:04.357319 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.357284    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-socket\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.357505 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.357328    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-podinfo\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.357505 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.357438    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-envoy\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.357597 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.357504    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istiod-ca-cert\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.357597 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.357525    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-data\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.357597 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.357547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-certs\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.357749 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.357614    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh24l\" (UniqueName: \"kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-kube-api-access-rh24l\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.357749 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.357644    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-token\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.357749 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.357661    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-credential-socket\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.458947 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.458851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-podinfo\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.458947 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.458906    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-envoy\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.458950    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istiod-ca-cert\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.458968    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-data\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.458991    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-certs\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.459071    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh24l\" (UniqueName: \"kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-kube-api-access-rh24l\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.459147    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-token\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.459170    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-credential-socket\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.459203    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-socket\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459610 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.459479    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-certs\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459610 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.459503    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-data\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459839 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.459814    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-credential-socket\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459908 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.459846    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istiod-ca-cert\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.459908 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.459904    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-socket\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.461423 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.461402    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-envoy\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.461574 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.461558    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-podinfo\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.467156 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.467127    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh24l\" (UniqueName: \"kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-kube-api-access-rh24l\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.467261 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.467136    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-token\") pod \"e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") " pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.595219 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.595181    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:04.781237 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.781154    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:04.877485 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.877453    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"]
Sep 11 16:57:04.879863 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:57:04.879828    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c2a64bf_f961_4dd4_a7cb_7e41d9bbe1ef.slice/crio-6c7e0913a4200ae464f93669b4471cfdd0631babc73e4fc5995e95c285460d74 WatchSource:0}: Error finding container 6c7e0913a4200ae464f93669b4471cfdd0631babc73e4fc5995e95c285460d74: Status 404 returned error can't find the container with id 6c7e0913a4200ae464f93669b4471cfdd0631babc73e4fc5995e95c285460d74
Sep 11 16:57:04.879863 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.879847    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:04.882211 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.882183    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:57:04.882320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.882257    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:57:04.882320 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:04.882299    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:57:05.509234 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:05.509194    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" event={"ID":"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef","Type":"ContainerStarted","Data":"d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e"}
Sep 11 16:57:05.509617 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:05.509259    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" event={"ID":"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef","Type":"ContainerStarted","Data":"6c7e0913a4200ae464f93669b4471cfdd0631babc73e4fc5995e95c285460d74"}
Sep 11 16:57:05.533000 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:05.532949    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" podStartSLOduration=1.532933001 podStartE2EDuration="1.532933001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:57:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:57:05.530131332 +0000 UTC m=+821.937349995" watchObservedRunningTime="2026-09-11 16:57:05.532933001 +0000 UTC m=+821.940151651"
Sep 11 16:57:05.595684 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:05.595647    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:05.601231 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:05.601203    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:05.622608 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:05.622584    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:06.407208 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:06.407178    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:06.515228 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:06.515192    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:06.516306 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:06.516288    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:06.948039 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:06.948009    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:07.035212 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:07.035185    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:07.469109 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:07.469044    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:08.141712 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.141676    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:08.753182 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.753154    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:57:08.906879 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.906845    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:08.910599 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.910568    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-token\") pod \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") "
Sep 11 16:57:08.910599 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.910603    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-data\") pod \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") "
Sep 11 16:57:08.910829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.910644    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lv8n2\" (UniqueName: \"kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-kube-api-access-lv8n2\") pod \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") "
Sep 11 16:57:08.910829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.910670    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-credential-socket\") pod \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") "
Sep 11 16:57:08.910829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.910714    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-podinfo\") pod \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") "
Sep 11 16:57:08.910829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.910756    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-certs\") pod \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") "
Sep 11 16:57:08.910829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.910779    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-socket\") pod \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") "
Sep 11 16:57:08.911127 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.910842    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istiod-ca-cert\") pod \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") "
Sep 11 16:57:08.911127 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.910878    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-envoy\") pod \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\" (UID: \"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1\") "
Sep 11 16:57:08.911127 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.910992    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" (UID: "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:57:08.911127 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.910997    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-data" (OuterVolumeSpecName: "istio-data") pod "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" (UID: "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:57:08.911345 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.911198    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" (UID: "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:57:08.911345 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.911319    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" (UID: "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:57:08.911460 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.911374    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:08.911460 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.911393    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:08.911460 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.911409    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:08.911460 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.911406    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" (UID: "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:57:08.912990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.912967    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" (UID: "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:57:08.913445 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.913208    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" (UID: "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:57:08.913554 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.913455    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-token" (OuterVolumeSpecName: "istio-token") pod "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" (UID: "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:57:08.913554 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:08.913485    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-kube-api-access-lv8n2" (OuterVolumeSpecName: "kube-api-access-lv8n2") pod "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" (UID: "fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1"). InnerVolumeSpecName "kube-api-access-lv8n2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:57:09.010791 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.010763    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:09.012066 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.012047    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:09.012147 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.012069    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:09.012147 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.012096    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lv8n2\" (UniqueName: \"kubernetes.io/projected/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-kube-api-access-lv8n2\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:09.012147 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.012110    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:09.012147 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.012123    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:09.012147 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.012133    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:09.118660 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.118627    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:09.425019 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.424980    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:09.532813 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.532777    2569 generic.go:356] "Generic (PLEG): container finished" podID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerID="11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec" exitCode=0
Sep 11 16:57:09.533024 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.532830    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"
Sep 11 16:57:09.533024 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.532885    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" event={"ID":"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1","Type":"ContainerDied","Data":"11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec"}
Sep 11 16:57:09.533024 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.532958    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt" event={"ID":"fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1","Type":"ContainerDied","Data":"7d18a175f9556c553a7c006f3a553000668ab59af204a9620375d37e8424228e"}
Sep 11 16:57:09.533024 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.532988    2569 scope.go:117] "RemoveContainer" containerID="11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec"
Sep 11 16:57:09.533482 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.533454    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:09.543114 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.543095    2569 scope.go:117] "RemoveContainer" containerID="ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c"
Sep 11 16:57:09.551634 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.551617    2569 scope.go:117] "RemoveContainer" containerID="11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec"
Sep 11 16:57:09.551896 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:57:09.551878    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec\": container with ID starting with 11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec not found: ID does not exist" containerID="11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec"
Sep 11 16:57:09.551956 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.551904    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec"} err="failed to get container status \"11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec\": rpc error: code = NotFound desc = could not find container \"11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec\": container with ID starting with 11e9361adb8ee9285ccde13f42b42b427808e64d51e5ca59f59656d3010be9ec not found: ID does not exist"
Sep 11 16:57:09.551956 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.551921    2569 scope.go:117] "RemoveContainer" containerID="ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c"
Sep 11 16:57:09.552200 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:57:09.552182    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c\": container with ID starting with ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c not found: ID does not exist" containerID="ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c"
Sep 11 16:57:09.552244 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.552206    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c"} err="failed to get container status \"ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c\": rpc error: code = NotFound desc = could not find container \"ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c\": container with ID starting with ff67878818ed024286f2b193365a0ae6ffd1256fe8a503aa79a1c1cefe5fc62c not found: ID does not exist"
Sep 11 16:57:09.563835 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.563809    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"]
Sep 11 16:57:09.577514 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.577492    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-5969a028-gw-openshift-default-68d6f98664-fvdnt"]
Sep 11 16:57:09.701290 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:09.701212    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:57:10.056252 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.056155    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:10.076327 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.076287    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"]
Sep 11 16:57:10.076730 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.076717    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy"
Sep 11 16:57:10.076772 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.076732    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy"
Sep 11 16:57:10.076772 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.076748    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy"
Sep 11 16:57:10.076772 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.076754    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy"
Sep 11 16:57:10.076869 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.076819    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy"
Sep 11 16:57:10.076869 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.076826    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" containerName="istio-proxy"
Sep 11 16:57:10.081336 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.081310    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.084023 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.083994    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2dc95c\"/\"simulator-w4-2dc95c-kserve-self-signed-certs\""
Sep 11 16:57:10.084023 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.084022    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2dc95c\"/\"default-dockercfg-rnfvr\""
Sep 11 16:57:10.084252 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.084092    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-2dc95c\"/\"openshift-service-ca.crt\""
Sep 11 16:57:10.084252 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.084092    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-2dc95c\"/\"kube-root-ca.crt\""
Sep 11 16:57:10.091657 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.091630    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"]
Sep 11 16:57:10.194216 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.194187    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:10.205922 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.205888    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1" path="/var/lib/kubelet/pods/fc4eaecf-2c80-4660-8d05-9d6a4c24d6d1/volumes"
Sep 11 16:57:10.225173 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.225133    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-model-cache\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.225363 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.225189    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh8tt\" (UniqueName: \"kubernetes.io/projected/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-kube-api-access-bh8tt\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.225363 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.225240    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-dshm\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.225363 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.225263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-home\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.225363 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.225309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tmp-dir\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.225537 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.225422    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tls-certs\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.326143 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.326019    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-model-cache\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.326143 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.326112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8tt\" (UniqueName: \"kubernetes.io/projected/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-kube-api-access-bh8tt\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.326358 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.326159    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-dshm\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.326358 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.326182    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-home\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.326358 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.326263    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tmp-dir\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.326510 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.326374    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tls-certs\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.326510 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.326451    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-model-cache\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.326790 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.326762    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tmp-dir\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.326869 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.326780    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-home\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.328530 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.328511    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-dshm\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.328916 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.328898    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tls-certs\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.334353 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.334331    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh8tt\" (UniqueName: \"kubernetes.io/projected/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-kube-api-access-bh8tt\") pod \"simulator-w4-2dc95c-kserve-54755b99c5-jd9b7\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") " pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.392176 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.392136    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:10.669970 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.669833    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"]
Sep 11 16:57:10.672795 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:57:10.672768    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1ac9601_61bc_4d88_ad5b_6a51a52c301f.slice/crio-212f5117a3fe62638b5f06eed2b80c2b44d895bf7f6e8e6a7322450a5340ad28 WatchSource:0}: Error finding container 212f5117a3fe62638b5f06eed2b80c2b44d895bf7f6e8e6a7322450a5340ad28: Status 404 returned error can't find the container with id 212f5117a3fe62638b5f06eed2b80c2b44d895bf7f6e8e6a7322450a5340ad28
Sep 11 16:57:10.803063 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:10.803031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:11.563370 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:11.563334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7" event={"ID":"b1ac9601-61bc-4d88-ad5b-6a51a52c301f","Type":"ContainerStarted","Data":"316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766"}
Sep 11 16:57:11.563589 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:11.563384    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7" event={"ID":"b1ac9601-61bc-4d88-ad5b-6a51a52c301f","Type":"ContainerStarted","Data":"212f5117a3fe62638b5f06eed2b80c2b44d895bf7f6e8e6a7322450a5340ad28"}
Sep 11 16:57:11.582406 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:11.582344    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7" podStartSLOduration=1.582322235 podStartE2EDuration="1.582322235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:57:11.581899873 +0000 UTC m=+827.989118548" watchObservedRunningTime="2026-09-11 16:57:11.582322235 +0000 UTC m=+827.989540886"
Sep 11 16:57:11.650356 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:11.650315    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:12.735281 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:12.735248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:17.191315 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:17.191274    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:20.392315 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:20.392276    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:20.392315 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:20.392327    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:20.405159 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:20.405130    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:20.625697 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:20.625668    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 16:57:28.580413 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:28.580321    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:57:28.671103 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:28.671044    2569 generic.go:356] "Generic (PLEG): container finished" podID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerID="d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e" exitCode=0
Sep 11 16:57:28.671269 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:28.671140    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" event={"ID":"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef","Type":"ContainerDied","Data":"d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e"}
Sep 11 16:57:28.671584 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:28.671570    2569 scope.go:117] "RemoveContainer" containerID="d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e"
Sep 11 16:57:28.672829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:28.672805    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:57:28.672889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:28.672870    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:57:28.672925 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:28.672903    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:57:29.678457 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:29.678421    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" event={"ID":"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef","Type":"ContainerStarted","Data":"3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603"}
Sep 11 16:57:29.827337 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:29.827271    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:30.595925 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:30.595883    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:30.596184 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:30.596164    2569 patch_prober.go:28] interesting pod/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 11 16:57:30.596240 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:30.596211    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 11 16:57:31.167482 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:31.167432    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:31.596437 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:31.596352    2569 patch_prober.go:28] interesting pod/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 11 16:57:31.596437 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:31.596402    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 11 16:57:31.999687 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:31.999653    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:32.595678 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:32.595641    2569 patch_prober.go:28] interesting pod/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 11 16:57:32.596107 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:32.595695    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 11 16:57:32.694422 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:32.694388    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:32.793886 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:32.793850    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"]
Sep 11 16:57:32.796529 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:32.796513    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"
Sep 11 16:57:32.798477 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:32.798456    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:57:32.798606 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:32.798480    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 16:57:32.804993 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:32.804965    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"]
Sep 11 16:57:32.889281 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:32.889247    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac0f6c5-8fb3-44bc-9374-7930bde27718-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx\" (UID: \"4ac0f6c5-8fb3-44bc-9374-7930bde27718\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"
Sep 11 16:57:32.990889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:32.990852    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac0f6c5-8fb3-44bc-9374-7930bde27718-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx\" (UID: \"4ac0f6c5-8fb3-44bc-9374-7930bde27718\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"
Sep 11 16:57:32.991512 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:32.991490    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac0f6c5-8fb3-44bc-9374-7930bde27718-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx\" (UID: \"4ac0f6c5-8fb3-44bc-9374-7930bde27718\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"
Sep 11 16:57:33.107846 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:33.107815    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"
Sep 11 16:57:33.404343 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:33.401248    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"]
Sep 11 16:57:33.565654 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:33.565625    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:33.595670 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:33.595635    2569 patch_prober.go:28] interesting pod/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 11 16:57:33.595837 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:33.595692    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 11 16:57:33.702784 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:33.702691    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx" event={"ID":"4ac0f6c5-8fb3-44bc-9374-7930bde27718","Type":"ContainerStarted","Data":"5a5e873e80b747cecde57dbb4214db4f4b6647b5270ce508d1ef22c31dc72a1f"}
Sep 11 16:57:34.443449 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.443406    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:34.596212 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.596125    2569 patch_prober.go:28] interesting pod/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 11 16:57:34.596212 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.596181    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 11 16:57:34.682556 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.682524    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"]
Sep 11 16:57:34.685164 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.685146    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.687267 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.687241    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-8b1a16\"/\"openshift-service-ca.crt\""
Sep 11 16:57:34.687832 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.687814    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-8b1a16\"/\"default-dockercfg-rvtx6\""
Sep 11 16:57:34.687925 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.687818    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-8b1a16\"/\"autores-model-8b1a16-kserve-self-signed-certs\""
Sep 11 16:57:34.687925 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.687890    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-8b1a16\"/\"kube-root-ca.crt\""
Sep 11 16:57:34.696487 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.696462    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"]
Sep 11 16:57:34.701736 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.701714    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:34.710172 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.710148    2569 generic.go:356] "Generic (PLEG): container finished" podID="4ac0f6c5-8fb3-44bc-9374-7930bde27718" containerID="c0decf13a2bee119bff984334b019d6406d7c2e9d1ef8dcba01327a9d47cd00a" exitCode=0
Sep 11 16:57:34.710273 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.710235    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx" event={"ID":"4ac0f6c5-8fb3-44bc-9374-7930bde27718","Type":"ContainerDied","Data":"c0decf13a2bee119bff984334b019d6406d7c2e9d1ef8dcba01327a9d47cd00a"}
Sep 11 16:57:34.810156 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.810120    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-tmp-dir\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.810348 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.810208    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-home\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.810348 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.810236    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-dshm\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.810348 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.810279    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-model-cache\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.810470 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.810385    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6rnc\" (UniqueName: \"kubernetes.io/projected/f2fca529-fd03-4f0b-8768-391602e78875-kube-api-access-t6rnc\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.810470 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.810465    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fca529-fd03-4f0b-8768-391602e78875-tls-certs\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.911855 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.911826    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6rnc\" (UniqueName: \"kubernetes.io/projected/f2fca529-fd03-4f0b-8768-391602e78875-kube-api-access-t6rnc\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.912058 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.911892    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fca529-fd03-4f0b-8768-391602e78875-tls-certs\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.912058 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.911924    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-tmp-dir\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.912058 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.911956    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-home\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.912058 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.911980    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-dshm\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.912058 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.912016    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-model-cache\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.912454 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.912427    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-home\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.912521 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.912434    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-model-cache\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.912521 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.912475    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-tmp-dir\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.914291 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.914271    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-dshm\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.914382 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.914363    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fca529-fd03-4f0b-8768-391602e78875-tls-certs\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.920373 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.920352    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6rnc\" (UniqueName: \"kubernetes.io/projected/f2fca529-fd03-4f0b-8768-391602e78875-kube-api-access-t6rnc\") pod \"autores-model-8b1a16-kserve-79cfd975dc-84clr\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") " pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:34.994828 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:34.994798    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:35.169465 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:35.169369    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:35.267163 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:35.267134    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"]
Sep 11 16:57:35.269651 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:57:35.269620    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2fca529_fd03_4f0b_8768_391602e78875.slice/crio-d048a5da64d4114775f7224af50e21c81a65ab06a6a92eb07bb2649f6d0efcc8 WatchSource:0}: Error finding container d048a5da64d4114775f7224af50e21c81a65ab06a6a92eb07bb2649f6d0efcc8: Status 404 returned error can't find the container with id d048a5da64d4114775f7224af50e21c81a65ab06a6a92eb07bb2649f6d0efcc8
Sep 11 16:57:35.600328 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:35.600243    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:35.600779 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:35.600636    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:35.601253 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:35.601235    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:35.717410 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:35.717375    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr" event={"ID":"f2fca529-fd03-4f0b-8768-391602e78875","Type":"ContainerStarted","Data":"08d0fc4e19312b104eefcc593915ae8a8d8257e7da1c8911c3fe3fbc615a6b86"}
Sep 11 16:57:35.717577 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:35.717417    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr" event={"ID":"f2fca529-fd03-4f0b-8768-391602e78875","Type":"ContainerStarted","Data":"d048a5da64d4114775f7224af50e21c81a65ab06a6a92eb07bb2649f6d0efcc8"}
Sep 11 16:57:35.749732 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:35.749681    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr" podStartSLOduration=1.749663505 podStartE2EDuration="1.749663505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:57:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:57:35.746386625 +0000 UTC m=+852.153605297" watchObservedRunningTime="2026-09-11 16:57:35.749663505 +0000 UTC m=+852.156882144"
Sep 11 16:57:35.861996 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:35.861934    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"
Sep 11 16:57:35.923150 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:35.923116    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac0f6c5-8fb3-44bc-9374-7930bde27718-maas-api-service-ca\") pod \"4ac0f6c5-8fb3-44bc-9374-7930bde27718\" (UID: \"4ac0f6c5-8fb3-44bc-9374-7930bde27718\") "
Sep 11 16:57:35.923435 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:35.923413    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ac0f6c5-8fb3-44bc-9374-7930bde27718-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ac0f6c5-8fb3-44bc-9374-7930bde27718" (UID: "4ac0f6c5-8fb3-44bc-9374-7930bde27718"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:57:36.024474 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:36.024438    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac0f6c5-8fb3-44bc-9374-7930bde27718-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:36.175166 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:36.175130    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:36.724753 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:36.724720    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"
Sep 11 16:57:36.725227 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:36.724763    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx" event={"ID":"4ac0f6c5-8fb3-44bc-9374-7930bde27718","Type":"ContainerDied","Data":"5a5e873e80b747cecde57dbb4214db4f4b6647b5270ce508d1ef22c31dc72a1f"}
Sep 11 16:57:36.725227 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:36.724839    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a5e873e80b747cecde57dbb4214db4f4b6647b5270ce508d1ef22c31dc72a1f"
Sep 11 16:57:36.954544 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:36.954495    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:37.967429 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:37.967385    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:38.671168 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.671134    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:38.863150 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.863112    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"]
Sep 11 16:57:38.863617 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.863600    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ac0f6c5-8fb3-44bc-9374-7930bde27718" containerName="revoke-keys"
Sep 11 16:57:38.863770 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.863620    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ac0f6c5-8fb3-44bc-9374-7930bde27718" containerName="revoke-keys"
Sep 11 16:57:38.863770 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.863750    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ac0f6c5-8fb3-44bc-9374-7930bde27718" containerName="revoke-keys"
Sep 11 16:57:38.865829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.865808    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:38.868045 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.868023    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4268e3\"/\"openshift-service-ca.crt\""
Sep 11 16:57:38.868186 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.868043    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4268e3\"/\"kube-root-ca.crt\""
Sep 11 16:57:38.868876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.868856    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4268e3\"/\"test-model-4268e3-kserve-self-signed-certs\""
Sep 11 16:57:38.868967 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.868864    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4268e3\"/\"default-dockercfg-js646\""
Sep 11 16:57:38.877599 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.877579    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"]
Sep 11 16:57:38.898768 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.898742    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:38.963164 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.963069    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-dshm\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:38.963309 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.963201    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrgxg\" (UniqueName: \"kubernetes.io/projected/c663f2c4-c71c-434e-acdb-490be6e886ac-kube-api-access-vrgxg\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:38.963309 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.963235    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-home\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:38.963389 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.963306    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-model-cache\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:38.963389 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.963327    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-tmp-dir\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:38.963455 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:38.963385    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c663f2c4-c71c-434e-acdb-490be6e886ac-tls-certs\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.064451 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.064416    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-dshm\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.064889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.064477    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrgxg\" (UniqueName: \"kubernetes.io/projected/c663f2c4-c71c-434e-acdb-490be6e886ac-kube-api-access-vrgxg\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.064889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.064498    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-home\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.064889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.064527    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-model-cache\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.064889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.064542    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-tmp-dir\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.064889 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.064571    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c663f2c4-c71c-434e-acdb-490be6e886ac-tls-certs\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.065150 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.064945    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-model-cache\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.065150 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.064947    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-home\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.065150 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.064993    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-tmp-dir\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.066699 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.066676    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-dshm\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.067026 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.067011    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c663f2c4-c71c-434e-acdb-490be6e886ac-tls-certs\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.072338 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.072313    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrgxg\" (UniqueName: \"kubernetes.io/projected/c663f2c4-c71c-434e-acdb-490be6e886ac-kube-api-access-vrgxg\") pod \"test-model-4268e3-kserve-746c9d9fcc-kk744\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") " pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.175238 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.175208    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:39.302401 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.302368    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:39.403266 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.403233    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:39.449734 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.449699    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"]
Sep 11 16:57:39.453864 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:57:39.453837    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc663f2c4_c71c_434e_acdb_490be6e886ac.slice/crio-122d8a628f5787d3c6a5eeaa5b99e512c11337a9f0ce800a4ca297be13350ab0 WatchSource:0}: Error finding container 122d8a628f5787d3c6a5eeaa5b99e512c11337a9f0ce800a4ca297be13350ab0: Status 404 returned error can't find the container with id 122d8a628f5787d3c6a5eeaa5b99e512c11337a9f0ce800a4ca297be13350ab0
Sep 11 16:57:39.483570 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.483504    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"]
Sep 11 16:57:39.486126 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.486098    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:39.486399 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.486340    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d310aa61-wk9rx"]
Sep 11 16:57:39.550932 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.550899    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:39.745254 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.745166    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744" event={"ID":"c663f2c4-c71c-434e-acdb-490be6e886ac","Type":"ContainerStarted","Data":"511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c"}
Sep 11 16:57:39.745254 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.745209    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744" event={"ID":"c663f2c4-c71c-434e-acdb-490be6e886ac","Type":"ContainerStarted","Data":"122d8a628f5787d3c6a5eeaa5b99e512c11337a9f0ce800a4ca297be13350ab0"}
Sep 11 16:57:39.762512 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.762463    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744" podStartSLOduration=1.762447106 podStartE2EDuration="1.762447106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:57:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:57:39.761863142 +0000 UTC m=+856.169081810" watchObservedRunningTime="2026-09-11 16:57:39.762447106 +0000 UTC m=+856.169665753"
Sep 11 16:57:39.968142 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:39.968106    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:40.211239 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:40.207192    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ac0f6c5-8fb3-44bc-9374-7930bde27718" path="/var/lib/kubelet/pods/4ac0f6c5-8fb3-44bc-9374-7930bde27718/volumes"
Sep 11 16:57:40.696729 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:40.696692    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"]
Sep 11 16:57:40.696989 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:40.696948    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy" containerID="cri-o://3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603" gracePeriod=30
Sep 11 16:57:40.709935 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:40.709902    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:40.810247 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:40.810214    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:41.165032 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.164987    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:41.474364 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.474256    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"]
Sep 11 16:57:41.477569 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.477544    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.479966 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.479936    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9abacf77-openshift-default-dockercfg-7zbks\""
Sep 11 16:57:41.490282 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.490254    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"]
Sep 11 16:57:41.491444 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.491416    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:41.588223 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.588184    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:41.597537 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.597493    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79msn\" (UniqueName: \"kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-kube-api-access-79msn\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.597752 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.597659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-socket\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.597829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.597754    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-envoy\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.597829 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.597798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-token\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.597934 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.597833    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-certs\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.597934 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.597879    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-data\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.597934 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.597904    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16ad847b-ba0d-4722-a782-7c918fae0aa5-istiod-ca-cert\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.598121 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.597987    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-credential-socket\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.598121 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.598039    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-podinfo\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.687560 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.687528    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:41.698935 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.698885    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79msn\" (UniqueName: \"kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-kube-api-access-79msn\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699162 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699057    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-socket\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699236 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699163    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-envoy\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699236 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699194    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-token\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699236 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-certs\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699266    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-data\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16ad847b-ba0d-4722-a782-7c918fae0aa5-istiod-ca-cert\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699338    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-credential-socket\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699384    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-podinfo\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699626 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699482    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-socket\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699626 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699595    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-data\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699767 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699734    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-certs\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.699816 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.699791    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-credential-socket\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.700055 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.700036    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16ad847b-ba0d-4722-a782-7c918fae0aa5-istiod-ca-cert\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.701512 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.701488    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-envoy\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.701871 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.701853    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-podinfo\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.706672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.706648    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-token\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.706813 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.706794    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79msn\" (UniqueName: \"kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-kube-api-access-79msn\") pod \"e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") " pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:41.789936 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:41.789834    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:42.088512 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:42.088483    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"]
Sep 11 16:57:42.090359 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:57:42.090333    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ad847b_ba0d_4722_a782_7c918fae0aa5.slice/crio-857097f6a627233741c07a69b827611f8d8c7a288d854b41f81bb4b01eb634e4 WatchSource:0}: Error finding container 857097f6a627233741c07a69b827611f8d8c7a288d854b41f81bb4b01eb634e4: Status 404 returned error can't find the container with id 857097f6a627233741c07a69b827611f8d8c7a288d854b41f81bb4b01eb634e4
Sep 11 16:57:42.092788 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:42.092752    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:57:42.092895 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:42.092839    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:57:42.092895 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:42.092870    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:57:42.320249 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:42.320210    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:42.764656 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:42.764619    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" event={"ID":"16ad847b-ba0d-4722-a782-7c918fae0aa5","Type":"ContainerStarted","Data":"e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63"}
Sep 11 16:57:42.765066 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:42.764673    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" event={"ID":"16ad847b-ba0d-4722-a782-7c918fae0aa5","Type":"ContainerStarted","Data":"857097f6a627233741c07a69b827611f8d8c7a288d854b41f81bb4b01eb634e4"}
Sep 11 16:57:42.787965 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:42.787902    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" podStartSLOduration=1.7878828850000001 podStartE2EDuration="1.787882885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:57:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:57:42.786169825 +0000 UTC m=+859.193388497" watchObservedRunningTime="2026-09-11 16:57:42.787882885 +0000 UTC m=+859.195101534"
Sep 11 16:57:42.790151 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:42.790128    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:43.287709 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:43.287666    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:43.795926 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:43.795891    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:44.162686 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:44.162635    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:44.275464 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:44.275432    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:44.375496 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:44.375463    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:44.601634 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:44.601543    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:57:44.775804 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:44.775769    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:44.776833 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:44.776810    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:57:44.995876 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:44.995842    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:44.996292 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:44.995888    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:45.008368 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:45.008335    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:45.083824 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:45.083792    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:45.791976 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:45.791948    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:57:45.952385 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:45.952356    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:46.046019 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.045938    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-certs\") pod \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") "
Sep 11 16:57:46.046019 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.045986    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-data\") pod \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") "
Sep 11 16:57:46.046547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046032    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-socket\") pod \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") "
Sep 11 16:57:46.046547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046067    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-credential-socket\") pod \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") "
Sep 11 16:57:46.046547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046141    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-envoy\") pod \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") "
Sep 11 16:57:46.046547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046174    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh24l\" (UniqueName: \"kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-kube-api-access-rh24l\") pod \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") "
Sep 11 16:57:46.046547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046216    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istiod-ca-cert\") pod \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") "
Sep 11 16:57:46.046547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046314    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-podinfo\") pod \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") "
Sep 11 16:57:46.046547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046302    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" (UID: "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:57:46.046547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046346    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" (UID: "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:57:46.046547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046404    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-token\") pod \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\" (UID: \"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef\") "
Sep 11 16:57:46.046547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046404    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-data" (OuterVolumeSpecName: "istio-data") pod "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" (UID: "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:57:46.046547 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046417    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" (UID: "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:57:46.047049 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046585    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" (UID: "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:57:46.047049 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046927    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:46.047049 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046950    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:46.047049 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046966    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:46.047049 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046978    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:46.047049 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.046993    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:46.048506 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.048485    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" (UID: "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:57:46.048506 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.048493    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-kube-api-access-rh24l" (OuterVolumeSpecName: "kube-api-access-rh24l") pod "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" (UID: "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef"). InnerVolumeSpecName "kube-api-access-rh24l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:57:46.048880 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.048863    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" (UID: "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:57:46.048880 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.048868    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-token" (OuterVolumeSpecName: "istio-token") pod "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" (UID: "9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:57:46.126455 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.126418    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:46.148117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.148060    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:46.148117 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.148118    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh24l\" (UniqueName: \"kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-kube-api-access-rh24l\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:46.148345 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.148133    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:46.148345 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.148146    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:57:46.787867 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.787837    2569 generic.go:356] "Generic (PLEG): container finished" podID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerID="3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603" exitCode=0
Sep 11 16:57:46.788062 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.787883    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"
Sep 11 16:57:46.788062 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.787934    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" event={"ID":"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef","Type":"ContainerDied","Data":"3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603"}
Sep 11 16:57:46.788062 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.787995    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx" event={"ID":"9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef","Type":"ContainerDied","Data":"6c7e0913a4200ae464f93669b4471cfdd0631babc73e4fc5995e95c285460d74"}
Sep 11 16:57:46.788062 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.788028    2569 scope.go:117] "RemoveContainer" containerID="3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603"
Sep 11 16:57:46.797237 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.797219    2569 scope.go:117] "RemoveContainer" containerID="d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e"
Sep 11 16:57:46.806275 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.806208    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"]
Sep 11 16:57:46.807333 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.807310    2569 scope.go:117] "RemoveContainer" containerID="3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603"
Sep 11 16:57:46.807632 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:57:46.807613    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603\": container with ID starting with 3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603 not found: ID does not exist" containerID="3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603"
Sep 11 16:57:46.807709 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.807644    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603"} err="failed to get container status \"3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603\": rpc error: code = NotFound desc = could not find container \"3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603\": container with ID starting with 3baabb68345704e54d49d61b1d4911f1e3b2d635c3aa433c8fb89b56531c3603 not found: ID does not exist"
Sep 11 16:57:46.807709 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.807668    2569 scope.go:117] "RemoveContainer" containerID="d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e"
Sep 11 16:57:46.807957 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:57:46.807940    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e\": container with ID starting with d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e not found: ID does not exist" containerID="d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e"
Sep 11 16:57:46.808141 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.807962    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e"} err="failed to get container status \"d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e\": rpc error: code = NotFound desc = could not find container \"d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e\": container with ID starting with d1178a894f2a5a4cf05a0f142c357665a4b16665c377f7c77337a51f9740a79e not found: ID does not exist"
Sep 11 16:57:46.808978 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:46.808958    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d310aa61-openshift-default-6d6988b798-76tnx"]
Sep 11 16:57:47.079734 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:47.079645    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:47.789395 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:47.789335    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:48.206258 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:48.206224    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" path="/var/lib/kubelet/pods/9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef/volumes"
Sep 11 16:57:48.579292 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:48.579198    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:49.175593 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:49.175557    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:49.175771 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:49.175608    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:49.187962 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:49.187936    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:49.361040 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:49.361001    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:49.425895 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:49.425813    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:49.817055 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:49.816967    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 16:57:49.849131 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:49.849100    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:49.955511 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:49.955469    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:57:50.054845 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:57:50.054810    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:06.982720 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:06.982673    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:58:08.173128 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:08.173090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:08.985669 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:08.985625    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:09.725225 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:09.725187    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:09.923101 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:09.923045    2569 generic.go:356] "Generic (PLEG): container finished" podID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerID="e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63" exitCode=0
Sep 11 16:58:09.923316 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:09.923131    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" event={"ID":"16ad847b-ba0d-4722-a782-7c918fae0aa5","Type":"ContainerDied","Data":"e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63"}
Sep 11 16:58:09.923627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:09.923609    2569 scope.go:117] "RemoveContainer" containerID="e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63"
Sep 11 16:58:09.924816 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:09.924789    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:58:09.924912 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:09.924865    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:58:09.924969 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:09.924916    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:58:10.442561 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:10.442524    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:10.931487 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:10.931447    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" event={"ID":"16ad847b-ba0d-4722-a782-7c918fae0aa5","Type":"ContainerStarted","Data":"2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b"}
Sep 11 16:58:11.126466 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:11.126430    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:11.791062 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:11.791014    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:58:11.792291 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:11.792266    2569 patch_prober.go:28] interesting pod/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 16:58:11.792415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:11.792321    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 16:58:11.887048 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:11.887013    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:12.751168 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:12.751134    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:12.790429 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:12.790393    2569 patch_prober.go:28] interesting pod/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 16:58:12.790615 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:12.790453    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 16:58:13.562958 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:13.562924    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:13.790522 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:13.790482    2569 patch_prober.go:28] interesting pod/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 16:58:13.790924 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:13.790542    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 16:58:14.180681 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.180644    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"]
Sep 11 16:58:14.181130 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.181116    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy"
Sep 11 16:58:14.181191 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.181131    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy"
Sep 11 16:58:14.181228 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.181212    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy"
Sep 11 16:58:14.181228 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.181223    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy"
Sep 11 16:58:14.183423 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.183405    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"
Sep 11 16:58:14.185460 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.185436    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 16:58:14.185595 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.185436    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:58:14.191570 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.191544    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"]
Sep 11 16:58:14.351106 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.351050    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:14.373990 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.373924    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/932ae2fd-46a8-42c2-9b96-4e33aedff8a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g\" (UID: \"932ae2fd-46a8-42c2-9b96-4e33aedff8a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"
Sep 11 16:58:14.475740 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.475627    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/932ae2fd-46a8-42c2-9b96-4e33aedff8a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g\" (UID: \"932ae2fd-46a8-42c2-9b96-4e33aedff8a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"
Sep 11 16:58:14.476310 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.476291    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/932ae2fd-46a8-42c2-9b96-4e33aedff8a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g\" (UID: \"932ae2fd-46a8-42c2-9b96-4e33aedff8a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"
Sep 11 16:58:14.520013 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.519974    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"
Sep 11 16:58:14.776411 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.776318    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:58:14.791271 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.791233    2569 patch_prober.go:28] interesting pod/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 16:58:14.791642 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.791294    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 16:58:14.816255 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.816186    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"]
Sep 11 16:58:14.819573 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:58:14.819516    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod932ae2fd_46a8_42c2_9b96_4e33aedff8a4.slice/crio-ca68fbf0b3a1f5febf8a828da67ee02a28bb756bd3447e729c5942403c0c8aae WatchSource:0}: Error finding container ca68fbf0b3a1f5febf8a828da67ee02a28bb756bd3447e729c5942403c0c8aae: Status 404 returned error can't find the container with id ca68fbf0b3a1f5febf8a828da67ee02a28bb756bd3447e729c5942403c0c8aae
Sep 11 16:58:14.965410 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:14.965374    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g" event={"ID":"932ae2fd-46a8-42c2-9b96-4e33aedff8a4","Type":"ContainerStarted","Data":"ca68fbf0b3a1f5febf8a828da67ee02a28bb756bd3447e729c5942403c0c8aae"}
Sep 11 16:58:15.790728 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:15.790697    2569 patch_prober.go:28] interesting pod/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 16:58:15.790923 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:15.790748    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 16:58:15.800791 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:15.800758    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:15.846303 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:15.846123    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"]
Sep 11 16:58:15.877266 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:15.877237    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:15.973883 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:15.973850    2569 generic.go:356] "Generic (PLEG): container finished" podID="932ae2fd-46a8-42c2-9b96-4e33aedff8a4" containerID="8ff86a6bfa604519f42f27f2531858e0022181d4f1fc9c11ed79a13bf4597fe3" exitCode=0
Sep 11 16:58:15.974099 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:15.973938    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g" event={"ID":"932ae2fd-46a8-42c2-9b96-4e33aedff8a4","Type":"ContainerDied","Data":"8ff86a6bfa604519f42f27f2531858e0022181d4f1fc9c11ed79a13bf4597fe3"}
Sep 11 16:58:16.076385 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:16.076286    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:16.794383 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:16.794346    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:58:16.795396 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:16.795376    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:58:16.870431 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:16.870402    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:17.120606 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:17.120583    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"
Sep 11 16:58:17.200292 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:17.200260    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/932ae2fd-46a8-42c2-9b96-4e33aedff8a4-maas-api-service-ca\") pod \"932ae2fd-46a8-42c2-9b96-4e33aedff8a4\" (UID: \"932ae2fd-46a8-42c2-9b96-4e33aedff8a4\") "
Sep 11 16:58:17.200589 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:17.200566    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/932ae2fd-46a8-42c2-9b96-4e33aedff8a4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "932ae2fd-46a8-42c2-9b96-4e33aedff8a4" (UID: "932ae2fd-46a8-42c2-9b96-4e33aedff8a4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:58:17.302150 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:17.302107    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/932ae2fd-46a8-42c2-9b96-4e33aedff8a4-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:17.906107 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:17.906059    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:17.986533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:17.986509    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"
Sep 11 16:58:17.986533 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:17.986516    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g" event={"ID":"932ae2fd-46a8-42c2-9b96-4e33aedff8a4","Type":"ContainerDied","Data":"ca68fbf0b3a1f5febf8a828da67ee02a28bb756bd3447e729c5942403c0c8aae"}
Sep 11 16:58:17.986741 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:17.986561    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca68fbf0b3a1f5febf8a828da67ee02a28bb756bd3447e729c5942403c0c8aae"
Sep 11 16:58:18.669479 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:18.669451    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:19.554638 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:19.554601    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:20.476013 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:20.475979    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:23.182038 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:23.181998    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:58:26.187689 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:26.187651    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:26.287687 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:26.287651    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:26.381725 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:26.381693    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"]
Sep 11 16:58:26.386095 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:26.386056    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9abacf77-m7v9g"]
Sep 11 16:58:26.386567 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:26.386546    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:26.473880 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:26.473774    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:26.532862 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:26.532827    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"]
Sep 11 16:58:26.533200 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:26.533111    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy" containerID="cri-o://2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b" gracePeriod=30
Sep 11 16:58:26.559555 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:26.559526    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:26.845178 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:26.845100    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:27.243086 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:27.243044    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:27.340147 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:27.340114    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:27.457986 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:27.457951    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:27.642662 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:27.642626    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:28.211138 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:28.211102    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="932ae2fd-46a8-42c2-9b96-4e33aedff8a4" path="/var/lib/kubelet/pods/932ae2fd-46a8-42c2-9b96-4e33aedff8a4/volumes"
Sep 11 16:58:28.503236 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:28.503150    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:28.900222 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:28.900186    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:29.233330 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:29.233242    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:29.926160 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:29.926118    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:30.636984 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:30.636955    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:30.876528 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:30.876483    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr" podUID="f2fca529-fd03-4f0b-8768-391602e78875" containerName="main" containerID="cri-o://08d0fc4e19312b104eefcc593915ae8a8d8257e7da1c8911c3fe3fbc615a6b86" gracePeriod=45
Sep 11 16:58:31.066935 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.066901    2569 generic.go:356] "Generic (PLEG): container finished" podID="f2fca529-fd03-4f0b-8768-391602e78875" containerID="08d0fc4e19312b104eefcc593915ae8a8d8257e7da1c8911c3fe3fbc615a6b86" exitCode=0
Sep 11 16:58:31.067341 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.066970    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr" event={"ID":"f2fca529-fd03-4f0b-8768-391602e78875","Type":"ContainerDied","Data":"08d0fc4e19312b104eefcc593915ae8a8d8257e7da1c8911c3fe3fbc615a6b86"}
Sep 11 16:58:31.124297 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.124268    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:58:31.259672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.259578    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-dshm\") pod \"f2fca529-fd03-4f0b-8768-391602e78875\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") "
Sep 11 16:58:31.259672 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.259646    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-home\") pod \"f2fca529-fd03-4f0b-8768-391602e78875\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") "
Sep 11 16:58:31.259920 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.259689    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-model-cache\") pod \"f2fca529-fd03-4f0b-8768-391602e78875\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") "
Sep 11 16:58:31.259920 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.259747    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6rnc\" (UniqueName: \"kubernetes.io/projected/f2fca529-fd03-4f0b-8768-391602e78875-kube-api-access-t6rnc\") pod \"f2fca529-fd03-4f0b-8768-391602e78875\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") "
Sep 11 16:58:31.259920 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.259800    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fca529-fd03-4f0b-8768-391602e78875-tls-certs\") pod \"f2fca529-fd03-4f0b-8768-391602e78875\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") "
Sep 11 16:58:31.259920 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.259828    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-tmp-dir\") pod \"f2fca529-fd03-4f0b-8768-391602e78875\" (UID: \"f2fca529-fd03-4f0b-8768-391602e78875\") "
Sep 11 16:58:31.260165 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.259963    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-home" (OuterVolumeSpecName: "home") pod "f2fca529-fd03-4f0b-8768-391602e78875" (UID: "f2fca529-fd03-4f0b-8768-391602e78875"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:31.260165 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.260040    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-model-cache" (OuterVolumeSpecName: "model-cache") pod "f2fca529-fd03-4f0b-8768-391602e78875" (UID: "f2fca529-fd03-4f0b-8768-391602e78875"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:31.260165 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.260147    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f2fca529-fd03-4f0b-8768-391602e78875" (UID: "f2fca529-fd03-4f0b-8768-391602e78875"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:31.260676 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.260433    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.260676 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.260455    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.260676 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.260470    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.262216 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.262194    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2fca529-fd03-4f0b-8768-391602e78875-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f2fca529-fd03-4f0b-8768-391602e78875" (UID: "f2fca529-fd03-4f0b-8768-391602e78875"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:58:31.262349 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.262330    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-dshm" (OuterVolumeSpecName: "dshm") pod "f2fca529-fd03-4f0b-8768-391602e78875" (UID: "f2fca529-fd03-4f0b-8768-391602e78875"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:31.262394 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.262350    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2fca529-fd03-4f0b-8768-391602e78875-kube-api-access-t6rnc" (OuterVolumeSpecName: "kube-api-access-t6rnc") pod "f2fca529-fd03-4f0b-8768-391602e78875" (UID: "f2fca529-fd03-4f0b-8768-391602e78875"). InnerVolumeSpecName "kube-api-access-t6rnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:58:31.361874 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.361837    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6rnc\" (UniqueName: \"kubernetes.io/projected/f2fca529-fd03-4f0b-8768-391602e78875-kube-api-access-t6rnc\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.361874 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.361865    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fca529-fd03-4f0b-8768-391602e78875-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.361874 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.361882    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fca529-fd03-4f0b-8768-391602e78875-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.777528 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.777502    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:58:31.867154 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867010    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-envoy\") pod \"16ad847b-ba0d-4722-a782-7c918fae0aa5\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") "
Sep 11 16:58:31.867154 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867115    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-certs\") pod \"16ad847b-ba0d-4722-a782-7c918fae0aa5\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") "
Sep 11 16:58:31.867391 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867196    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79msn\" (UniqueName: \"kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-kube-api-access-79msn\") pod \"16ad847b-ba0d-4722-a782-7c918fae0aa5\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") "
Sep 11 16:58:31.867391 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867225    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-podinfo\") pod \"16ad847b-ba0d-4722-a782-7c918fae0aa5\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") "
Sep 11 16:58:31.867391 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867276    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-data\") pod \"16ad847b-ba0d-4722-a782-7c918fae0aa5\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") "
Sep 11 16:58:31.867391 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867322    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-credential-socket\") pod \"16ad847b-ba0d-4722-a782-7c918fae0aa5\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") "
Sep 11 16:58:31.867391 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867352    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-token\") pod \"16ad847b-ba0d-4722-a782-7c918fae0aa5\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") "
Sep 11 16:58:31.867391 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867382    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16ad847b-ba0d-4722-a782-7c918fae0aa5-istiod-ca-cert\") pod \"16ad847b-ba0d-4722-a782-7c918fae0aa5\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") "
Sep 11 16:58:31.867683 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867396    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "16ad847b-ba0d-4722-a782-7c918fae0aa5" (UID: "16ad847b-ba0d-4722-a782-7c918fae0aa5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:31.867683 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867470    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-socket\") pod \"16ad847b-ba0d-4722-a782-7c918fae0aa5\" (UID: \"16ad847b-ba0d-4722-a782-7c918fae0aa5\") "
Sep 11 16:58:31.867683 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867616    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "16ad847b-ba0d-4722-a782-7c918fae0aa5" (UID: "16ad847b-ba0d-4722-a782-7c918fae0aa5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:31.867853 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867824    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "16ad847b-ba0d-4722-a782-7c918fae0aa5" (UID: "16ad847b-ba0d-4722-a782-7c918fae0aa5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:31.867853 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867834    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ad847b-ba0d-4722-a782-7c918fae0aa5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "16ad847b-ba0d-4722-a782-7c918fae0aa5" (UID: "16ad847b-ba0d-4722-a782-7c918fae0aa5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:58:31.867945 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867895    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-data" (OuterVolumeSpecName: "istio-data") pod "16ad847b-ba0d-4722-a782-7c918fae0aa5" (UID: "16ad847b-ba0d-4722-a782-7c918fae0aa5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:31.867992 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867966    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.867992 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.867986    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.868093 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.868001    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.868093 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.868016    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16ad847b-ba0d-4722-a782-7c918fae0aa5-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.869438 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.869419    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "16ad847b-ba0d-4722-a782-7c918fae0aa5" (UID: "16ad847b-ba0d-4722-a782-7c918fae0aa5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:58:31.869539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.869520    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-token" (OuterVolumeSpecName: "istio-token") pod "16ad847b-ba0d-4722-a782-7c918fae0aa5" (UID: "16ad847b-ba0d-4722-a782-7c918fae0aa5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:58:31.869774 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.869757    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-kube-api-access-79msn" (OuterVolumeSpecName: "kube-api-access-79msn") pod "16ad847b-ba0d-4722-a782-7c918fae0aa5" (UID: "16ad847b-ba0d-4722-a782-7c918fae0aa5"). InnerVolumeSpecName "kube-api-access-79msn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:58:31.869824 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.869757    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "16ad847b-ba0d-4722-a782-7c918fae0aa5" (UID: "16ad847b-ba0d-4722-a782-7c918fae0aa5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:31.969352 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.969314    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.969352 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.969346    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.969352 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.969354    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.969589 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.969363    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79msn\" (UniqueName: \"kubernetes.io/projected/16ad847b-ba0d-4722-a782-7c918fae0aa5-kube-api-access-79msn\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.969589 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:31.969374    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16ad847b-ba0d-4722-a782-7c918fae0aa5-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:32.075057 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.075019    2569 generic.go:356] "Generic (PLEG): container finished" podID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerID="2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b" exitCode=0
Sep 11 16:58:32.075512 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.075070    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"
Sep 11 16:58:32.075512 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.075107    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" event={"ID":"16ad847b-ba0d-4722-a782-7c918fae0aa5","Type":"ContainerDied","Data":"2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b"}
Sep 11 16:58:32.075512 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.075164    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm" event={"ID":"16ad847b-ba0d-4722-a782-7c918fae0aa5","Type":"ContainerDied","Data":"857097f6a627233741c07a69b827611f8d8c7a288d854b41f81bb4b01eb634e4"}
Sep 11 16:58:32.075512 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.075194    2569 scope.go:117] "RemoveContainer" containerID="2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b"
Sep 11 16:58:32.076619 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.076582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr" event={"ID":"f2fca529-fd03-4f0b-8768-391602e78875","Type":"ContainerDied","Data":"d048a5da64d4114775f7224af50e21c81a65ab06a6a92eb07bb2649f6d0efcc8"}
Sep 11 16:58:32.076619 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.076601    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"
Sep 11 16:58:32.085054 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.085034    2569 scope.go:117] "RemoveContainer" containerID="e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63"
Sep 11 16:58:32.093857 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.093838    2569 scope.go:117] "RemoveContainer" containerID="2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b"
Sep 11 16:58:32.094154 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:58:32.094137    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b\": container with ID starting with 2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b not found: ID does not exist" containerID="2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b"
Sep 11 16:58:32.094206 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.094164    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b"} err="failed to get container status \"2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b\": rpc error: code = NotFound desc = could not find container \"2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b\": container with ID starting with 2a26c5721a736ffb0d3902171bcc12f6c46e7c0b1016400e7cfbeb7563202d4b not found: ID does not exist"
Sep 11 16:58:32.094206 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.094181    2569 scope.go:117] "RemoveContainer" containerID="e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63"
Sep 11 16:58:32.094436 ip-10-0-135-55 kubenswrapper[2569]: E0911 16:58:32.094417    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63\": container with ID starting with e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63 not found: ID does not exist" containerID="e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63"
Sep 11 16:58:32.094478 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.094442    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63"} err="failed to get container status \"e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63\": rpc error: code = NotFound desc = could not find container \"e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63\": container with ID starting with e39d285488768522739ca48fabbdb5e91db6fafc97c8e848b9f84cf23d48bb63 not found: ID does not exist"
Sep 11 16:58:32.094478 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.094458    2569 scope.go:117] "RemoveContainer" containerID="08d0fc4e19312b104eefcc593915ae8a8d8257e7da1c8911c3fe3fbc615a6b86"
Sep 11 16:58:32.100581 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.100559    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"]
Sep 11 16:58:32.105792 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.105769    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9abacf77-openshift-default-f9549dcd4-4mchm"]
Sep 11 16:58:32.117334 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.117264    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"]
Sep 11 16:58:32.120509 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.120489    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-8b1a16/autores-model-8b1a16-kserve-79cfd975dc-84clr"]
Sep 11 16:58:32.205415 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.205381    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" path="/var/lib/kubelet/pods/16ad847b-ba0d-4722-a782-7c918fae0aa5/volumes"
Sep 11 16:58:32.205817 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:32.205803    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2fca529-fd03-4f0b-8768-391602e78875" path="/var/lib/kubelet/pods/f2fca529-fd03-4f0b-8768-391602e78875/volumes"
Sep 11 16:58:35.146498 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:35.146463    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:35.262726 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:35.262691    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:35.348905 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:35.348870    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:35.448665 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:35.448579    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:35.971754 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:35.971724    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:38.160523 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:38.160484    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:38.780647 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:38.780614    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:38.875998 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:38.875962    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:38.934425 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:38.934374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:39.015240 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:39.015211    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:39.081312 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:39.081227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:39.394941 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:39.394905    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:39.495875 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:39.495842    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:40.323470 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:40.323429    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:41.286568 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:41.286528    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:42.099445 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:42.099410    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:42.893665 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:42.893629    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:43.788248 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:43.788212    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:58:45.799589 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:45.799555    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:45.899598 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:45.899564    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:46.051360 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.051273    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"]
Sep 11 16:58:46.051564 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.051538    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy" containerID="cri-o://cfc63baf6e8c7667b9f756a70fd7553f527a01aa0faf556cf45942c7470a2930" gracePeriod=30
Sep 11 16:58:46.077484 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.077448    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:46.526934 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.526902    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:46.823725 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.823640    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"]
Sep 11 16:58:46.824286 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824266    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2fca529-fd03-4f0b-8768-391602e78875" containerName="main"
Sep 11 16:58:46.824379 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824288    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2fca529-fd03-4f0b-8768-391602e78875" containerName="main"
Sep 11 16:58:46.824379 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824313    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy"
Sep 11 16:58:46.824379 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824321    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy"
Sep 11 16:58:46.824379 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824332    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy"
Sep 11 16:58:46.824379 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824342    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c2a64bf-f961-4dd4-a7cb-7e41d9bbe1ef" containerName="istio-proxy"
Sep 11 16:58:46.824379 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824354    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="932ae2fd-46a8-42c2-9b96-4e33aedff8a4" containerName="revoke-keys"
Sep 11 16:58:46.824379 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824362    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="932ae2fd-46a8-42c2-9b96-4e33aedff8a4" containerName="revoke-keys"
Sep 11 16:58:46.824708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824383    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy"
Sep 11 16:58:46.824708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824392    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy"
Sep 11 16:58:46.824708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824502    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="932ae2fd-46a8-42c2-9b96-4e33aedff8a4" containerName="revoke-keys"
Sep 11 16:58:46.824708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824515    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2fca529-fd03-4f0b-8768-391602e78875" containerName="main"
Sep 11 16:58:46.824708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824523    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy"
Sep 11 16:58:46.824708 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.824533    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="16ad847b-ba0d-4722-a782-7c918fae0aa5" containerName="istio-proxy"
Sep 11 16:58:46.829229 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.829208    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:46.831209 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.831190    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-2cceea-openshift-default-dockercfg-28cqq\""
Sep 11 16:58:46.839502 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.839478    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"]
Sep 11 16:58:46.840680 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.840657    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:46.934525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.933920    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfe259e3-e59a-4c41-a782-b1941356f8b9-istiod-ca-cert\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:46.934525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.934001    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-envoy\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:46.934525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.934033    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-credential-socket\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:46.934525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.934125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-certs\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:46.934525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.934173    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-podinfo\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:46.934525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.934199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k4rp\" (UniqueName: \"kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-kube-api-access-8k4rp\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:46.934525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.934251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-socket\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:46.934525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.934312    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-data\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:46.934525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:46.934385    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-token\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.035765 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.035734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-podinfo\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.035765 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.035768    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k4rp\" (UniqueName: \"kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-kube-api-access-8k4rp\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.035984 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.035804    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-socket\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.035984 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.035844    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-data\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.035984 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.035894    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-token\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.035984 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.035925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfe259e3-e59a-4c41-a782-b1941356f8b9-istiod-ca-cert\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.036202 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.035986    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-envoy\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.036202 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.036014    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-credential-socket\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.036202 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.036182    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-certs\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.036539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.036366    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-data\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.036539 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.036393    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-socket\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.036734 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.036542    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-certs\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.036734 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.036588    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-credential-socket\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.036854 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.036756    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfe259e3-e59a-4c41-a782-b1941356f8b9-istiod-ca-cert\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.038420 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.038393    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-podinfo\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.038568 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.038546    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-envoy\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.057551 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.057525    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-token\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.058885 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.058864    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k4rp\" (UniqueName: \"kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-kube-api-access-8k4rp\") pod \"e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") " pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.115519 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.115438    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:47.140086 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.140050    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:47.422722 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.422689    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"]
Sep 11 16:58:47.426210 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:58:47.426187    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfe259e3_e59a_4c41_a782_b1941356f8b9.slice/crio-d02734d2bc7ebcf9e0d4cedbb1f2e9e74b225240fe009e8ffa006d6978507c30 WatchSource:0}: Error finding container d02734d2bc7ebcf9e0d4cedbb1f2e9e74b225240fe009e8ffa006d6978507c30: Status 404 returned error can't find the container with id d02734d2bc7ebcf9e0d4cedbb1f2e9e74b225240fe009e8ffa006d6978507c30
Sep 11 16:58:47.428504 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.428472    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:58:47.428598 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.428540    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:58:47.428598 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:47.428570    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:58:48.183023 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:48.182985    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" event={"ID":"cfe259e3-e59a-4c41-a782-b1941356f8b9","Type":"ContainerStarted","Data":"dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f"}
Sep 11 16:58:48.183451 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:48.183050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" event={"ID":"cfe259e3-e59a-4c41-a782-b1941356f8b9","Type":"ContainerStarted","Data":"d02734d2bc7ebcf9e0d4cedbb1f2e9e74b225240fe009e8ffa006d6978507c30"}
Sep 11 16:58:48.202124 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:48.202033    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" podStartSLOduration=2.202012979 podStartE2EDuration="2.202012979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:58:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:58:48.200910724 +0000 UTC m=+924.608129390" watchObservedRunningTime="2026-09-11 16:58:48.202012979 +0000 UTC m=+924.609231627"
Sep 11 16:58:49.141182 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:49.141149    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:49.146530 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:49.146502    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:49.189635 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:49.189605    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:49.190521 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:49.190502    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:58:49.457019 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:49.456931    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:50.220768 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:50.220732    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:50.313525 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:50.313493    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:50.522680 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:50.522587    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:51.204997 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.204967    2569 generic.go:356] "Generic (PLEG): container finished" podID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerID="cfc63baf6e8c7667b9f756a70fd7553f527a01aa0faf556cf45942c7470a2930" exitCode=0
Sep 11 16:58:51.205202 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.205048    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" event={"ID":"15380047-35b2-4d02-8bb4-81a0c09a6519","Type":"ContainerDied","Data":"cfc63baf6e8c7667b9f756a70fd7553f527a01aa0faf556cf45942c7470a2930"}
Sep 11 16:58:51.205202 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.205129    2569 scope.go:117] "RemoveContainer" containerID="283931ece6996db539885cdc09f0d7ef887983e61b9f33d107a924f46e0382d0"
Sep 11 16:58:51.315064 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.314994    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:58:51.381467 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381436    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15380047-35b2-4d02-8bb4-81a0c09a6519-istiod-ca-cert\") pod \"15380047-35b2-4d02-8bb4-81a0c09a6519\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") "
Sep 11 16:58:51.381611 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381478    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-envoy\") pod \"15380047-35b2-4d02-8bb4-81a0c09a6519\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") "
Sep 11 16:58:51.381611 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381504    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-credential-socket\") pod \"15380047-35b2-4d02-8bb4-81a0c09a6519\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") "
Sep 11 16:58:51.381611 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381549    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-token\") pod \"15380047-35b2-4d02-8bb4-81a0c09a6519\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") "
Sep 11 16:58:51.381611 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381587    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-socket\") pod \"15380047-35b2-4d02-8bb4-81a0c09a6519\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") "
Sep 11 16:58:51.381809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381622    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-data\") pod \"15380047-35b2-4d02-8bb4-81a0c09a6519\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") "
Sep 11 16:58:51.381809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381648    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxs4s\" (UniqueName: \"kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-kube-api-access-gxs4s\") pod \"15380047-35b2-4d02-8bb4-81a0c09a6519\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") "
Sep 11 16:58:51.381809 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381791    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "15380047-35b2-4d02-8bb4-81a0c09a6519" (UID: "15380047-35b2-4d02-8bb4-81a0c09a6519"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:51.381963 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381807    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-podinfo\") pod \"15380047-35b2-4d02-8bb4-81a0c09a6519\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") "
Sep 11 16:58:51.381963 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381797    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15380047-35b2-4d02-8bb4-81a0c09a6519-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "15380047-35b2-4d02-8bb4-81a0c09a6519" (UID: "15380047-35b2-4d02-8bb4-81a0c09a6519"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:58:51.381963 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381897    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-certs\") pod \"15380047-35b2-4d02-8bb4-81a0c09a6519\" (UID: \"15380047-35b2-4d02-8bb4-81a0c09a6519\") "
Sep 11 16:58:51.382173 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.381964    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "15380047-35b2-4d02-8bb4-81a0c09a6519" (UID: "15380047-35b2-4d02-8bb4-81a0c09a6519"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:51.382173 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.382005    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-data" (OuterVolumeSpecName: "istio-data") pod "15380047-35b2-4d02-8bb4-81a0c09a6519" (UID: "15380047-35b2-4d02-8bb4-81a0c09a6519"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:51.382290 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.382232    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "15380047-35b2-4d02-8bb4-81a0c09a6519" (UID: "15380047-35b2-4d02-8bb4-81a0c09a6519"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:51.382562 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.382528    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:51.382562 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.382550    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:51.382733 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.382564    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15380047-35b2-4d02-8bb4-81a0c09a6519-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:51.382733 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.382578    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:51.382733 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.382592    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:51.383793 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.383770    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "15380047-35b2-4d02-8bb4-81a0c09a6519" (UID: "15380047-35b2-4d02-8bb4-81a0c09a6519"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:51.383890 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.383875    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-token" (OuterVolumeSpecName: "istio-token") pod "15380047-35b2-4d02-8bb4-81a0c09a6519" (UID: "15380047-35b2-4d02-8bb4-81a0c09a6519"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:58:51.383982 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.383961    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-kube-api-access-gxs4s" (OuterVolumeSpecName: "kube-api-access-gxs4s") pod "15380047-35b2-4d02-8bb4-81a0c09a6519" (UID: "15380047-35b2-4d02-8bb4-81a0c09a6519"). InnerVolumeSpecName "kube-api-access-gxs4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:58:51.384146 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.384067    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "15380047-35b2-4d02-8bb4-81a0c09a6519" (UID: "15380047-35b2-4d02-8bb4-81a0c09a6519"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:58:51.484289 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.484249    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxs4s\" (UniqueName: \"kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-kube-api-access-gxs4s\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:51.484289 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.484283    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:51.484289 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.484295    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:51.484527 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.484303    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15380047-35b2-4d02-8bb4-81a0c09a6519-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 16:58:51.486141 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:51.486107    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:52.212180 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:52.212151    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"
Sep 11 16:58:52.212377 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:52.212199    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb" event={"ID":"15380047-35b2-4d02-8bb4-81a0c09a6519","Type":"ContainerDied","Data":"01f3b609962fb444b4d375a253bca36756555f20575d477748190b78afce96c8"}
Sep 11 16:58:52.212377 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:52.212276    2569 scope.go:117] "RemoveContainer" containerID="cfc63baf6e8c7667b9f756a70fd7553f527a01aa0faf556cf45942c7470a2930"
Sep 11 16:58:52.229630 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:52.229601    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"]
Sep 11 16:58:52.235124 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:52.235096    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-8b1a16-openshift-default-787d566cdd-x4wrb"]
Sep 11 16:58:52.358516 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:52.358485    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:53.176521 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:53.176487    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:54.199530 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:54.199490    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:54.207218 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:54.207182    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" path="/var/lib/kubelet/pods/15380047-35b2-4d02-8bb4-81a0c09a6519/volumes"
Sep 11 16:58:54.300283 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:54.300251    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:54.552417 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:54.552328    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:54.685383 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:54.685351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:58:54.788035 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:58:54.788002    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:01.892986 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:01.892948    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:02.786879 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:02.786845    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:59:06.629718 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:06.629679    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:08.030244 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:08.030193    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:08.136367 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:08.136217    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:08.231071 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:08.231030    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:12.956335 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:12.956286    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:13.010252 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:13.010219    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:13.339727 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:13.339638    2569 generic.go:356] "Generic (PLEG): container finished" podID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerID="dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f" exitCode=0
Sep 11 16:59:13.339727 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:13.339711    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" event={"ID":"cfe259e3-e59a-4c41-a782-b1941356f8b9","Type":"ContainerDied","Data":"dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f"}
Sep 11 16:59:13.340193 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:13.340176    2569 scope.go:117] "RemoveContainer" containerID="dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f"
Sep 11 16:59:13.341430 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:13.341405    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:59:13.341498 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:13.341474    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:59:13.341537 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:13.341506    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 16:59:14.347552 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:14.347513    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" event={"ID":"cfe259e3-e59a-4c41-a782-b1941356f8b9","Type":"ContainerStarted","Data":"8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3"}
Sep 11 16:59:15.142332 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.142285    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:59:15.142574 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.142439    2569 patch_prober.go:28] interesting pod/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 11 16:59:15.142574 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.142501    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 11 16:59:15.878849 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.878815    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"]
Sep 11 16:59:15.879297 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.879283    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy"
Sep 11 16:59:15.879347 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.879299    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy"
Sep 11 16:59:15.879347 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.879315    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy"
Sep 11 16:59:15.879347 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.879320    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy"
Sep 11 16:59:15.879455 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.879395    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy"
Sep 11 16:59:15.879455 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.879402    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="15380047-35b2-4d02-8bb4-81a0c09a6519" containerName="istio-proxy"
Sep 11 16:59:15.882400 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.882382    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:15.884705 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.884681    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-52b0af\"/\"premium-w0-52b0af-kserve-self-signed-certs\""
Sep 11 16:59:15.893806 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.893773    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"]
Sep 11 16:59:15.909794 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.909758    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:15.955663 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.955624    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tls-certs\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:15.955663 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.955665    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-home\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:15.955894 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.955745    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwgz7\" (UniqueName: \"kubernetes.io/projected/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-kube-api-access-vwgz7\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:15.955894 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.955788    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-dshm\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:15.955894 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.955872    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-model-cache\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:15.956015 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:15.955896    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tmp-dir\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.056916 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.056882    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-model-cache\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.056916 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.056923    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tmp-dir\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.057213 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.056999    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tls-certs\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.057213 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.057041    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-home\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.057213 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.057097    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwgz7\" (UniqueName: \"kubernetes.io/projected/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-kube-api-access-vwgz7\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.057213 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.057147    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-dshm\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.057369 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.057300    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-model-cache\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.057580 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.057558    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tmp-dir\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.057655 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.057558    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-home\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.059488 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.059468    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-dshm\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.059916 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.059893    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tls-certs\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.065318 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.065294    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwgz7\" (UniqueName: \"kubernetes.io/projected/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-kube-api-access-vwgz7\") pod \"premium-w0-52b0af-kserve-75c6ffdb-f7dn9\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") " pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.140653 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.140544    2569 patch_prober.go:28] interesting pod/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 11 16:59:16.140653 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.140623    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 11 16:59:16.183492 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.183460    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:16.192434 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.192408    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:16.496763 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.496734    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"]
Sep 11 16:59:16.498638 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:59:16.498602    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e80c4fb_23a2_41f8_95ed_d5b5cbc856ce.slice/crio-98165028a70b4f906529e9b55e29ebe974fdb3f0e9d32e9cd35490d44e7cc454 WatchSource:0}: Error finding container 98165028a70b4f906529e9b55e29ebe974fdb3f0e9d32e9cd35490d44e7cc454: Status 404 returned error can't find the container with id 98165028a70b4f906529e9b55e29ebe974fdb3f0e9d32e9cd35490d44e7cc454
Sep 11 16:59:16.977941 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:16.977892    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:17.141515 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:17.141465    2569 patch_prober.go:28] interesting pod/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 11 16:59:17.141780 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:17.141546    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 11 16:59:17.376524 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:17.376489    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9" event={"ID":"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce","Type":"ContainerStarted","Data":"0d196dcd6a89b4a6fb8d5d7c03df99a9f2687c9003ee410ab05207eb91ddd2ec"}
Sep 11 16:59:17.376729 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:17.376531    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9" event={"ID":"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce","Type":"ContainerStarted","Data":"98165028a70b4f906529e9b55e29ebe974fdb3f0e9d32e9cd35490d44e7cc454"}
Sep 11 16:59:17.395094 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:17.395017    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9" podStartSLOduration=2.39499539 podStartE2EDuration="2.39499539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:59:17.392686069 +0000 UTC m=+953.799904729" watchObservedRunningTime="2026-09-11 16:59:17.39499539 +0000 UTC m=+953.802214039"
Sep 11 16:59:17.789567 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:17.789485    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:18.140573 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:18.140527    2569 patch_prober.go:28] interesting pod/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 11 16:59:18.141108 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:18.140599    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 11 16:59:18.506654 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:18.506569    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:19.140699 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:19.140652    2569 patch_prober.go:28] interesting pod/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 11 16:59:19.141231 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:19.140723    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 11 16:59:19.190467 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:19.190440    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:59:19.264684 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:19.264647    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:19.973391 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:19.973356    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:20.141501 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:20.141461    2569 patch_prober.go:28] interesting pod/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 11 16:59:20.142022 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:20.141535    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 11 16:59:20.175302 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:20.175269    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:21.144693 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:21.144664    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:59:21.145678 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:21.145657    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 16:59:23.231567 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:23.231533    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:59:24.073298 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:24.073265    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:26.193223 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.193183    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:26.193628 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.193238    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:26.206373 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.206347    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:26.440956 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.440923    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"]
Sep 11 16:59:26.446017 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.445961    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 16:59:26.446147 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.446038    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.449835 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.449821    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2dc95c\"/\"premium-w4-2dc95c-kserve-self-signed-certs\""
Sep 11 16:59:26.473850 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.473820    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"]
Sep 11 16:59:26.485752 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.485719    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:26.567564 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.567531    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ab41a6-a42e-4f81-976f-28317bfc1181-tls-certs\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.567731 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.567598    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-model-cache\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.567731 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.567675    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-dshm\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.567731 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.567693    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-tmp-dir\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.567731 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.567722    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-home\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.567873 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.567809    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clv45\" (UniqueName: \"kubernetes.io/projected/92ab41a6-a42e-4f81-976f-28317bfc1181-kube-api-access-clv45\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.668397 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.668363    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clv45\" (UniqueName: \"kubernetes.io/projected/92ab41a6-a42e-4f81-976f-28317bfc1181-kube-api-access-clv45\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.668568 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.668406    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ab41a6-a42e-4f81-976f-28317bfc1181-tls-certs\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.668568 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.668461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-model-cache\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.668568 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.668521    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-dshm\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.668568 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.668539    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-tmp-dir\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.668767 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.668572    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-home\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.669090 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.669045    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-tmp-dir\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.669185 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.669113    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-home\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.669185 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.669055    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-model-cache\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.670897 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.670878    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-dshm\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.671216 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.671198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ab41a6-a42e-4f81-976f-28317bfc1181-tls-certs\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.677045 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.677011    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clv45\" (UniqueName: \"kubernetes.io/projected/92ab41a6-a42e-4f81-976f-28317bfc1181-kube-api-access-clv45\") pod \"premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") " pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:26.755382 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:26.755311    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:27.033354 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:27.033321    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"]
Sep 11 16:59:27.033706 ip-10-0-135-55 kubenswrapper[2569]: W0911 16:59:27.033675    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92ab41a6_a42e_4f81_976f_28317bfc1181.slice/crio-516759d823f8abfd42daf57739b1ed702296a84ec3cca017dec094812ba34b9b WatchSource:0}: Error finding container 516759d823f8abfd42daf57739b1ed702296a84ec3cca017dec094812ba34b9b: Status 404 returned error can't find the container with id 516759d823f8abfd42daf57739b1ed702296a84ec3cca017dec094812ba34b9b
Sep 11 16:59:27.438495 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:27.438453    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2" event={"ID":"92ab41a6-a42e-4f81-976f-28317bfc1181","Type":"ContainerStarted","Data":"f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c"}
Sep 11 16:59:27.438912 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:27.438506    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2" event={"ID":"92ab41a6-a42e-4f81-976f-28317bfc1181","Type":"ContainerStarted","Data":"516759d823f8abfd42daf57739b1ed702296a84ec3cca017dec094812ba34b9b"}
Sep 11 16:59:27.455675 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:27.455626    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2" podStartSLOduration=1.455610756 podStartE2EDuration="1.455610756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:59:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:59:27.453434039 +0000 UTC m=+963.860652699" watchObservedRunningTime="2026-09-11 16:59:27.455610756 +0000 UTC m=+963.862829404"
Sep 11 16:59:30.267714 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:30.267683    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:30.337319 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:30.337270    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:30.431466 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:30.431434    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:30.531223 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:30.531135    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:36.756023 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:36.755984    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:36.756483 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:36.756044    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:36.768834 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:36.768809    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:37.510430 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:37.510401    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 16:59:40.098994 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:40.098962    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:40.680627 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:40.680591    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 16:59:40.971467 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:40.971375    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:41.773756 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:41.773710    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:44.717371 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:44.717327    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:48.123715 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:48.123635    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:48.179711 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:48.179675    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:48.268868 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:48.268813    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:48.362150 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:48.362118    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:48.468640 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:48.468605    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:49.099284 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:49.099250    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:49.179897 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:49.179861    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:49.280187 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:49.280147    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:50.735243 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:50.735205    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:56.921316 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:56.921280    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:56.985913 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:56.985878    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:57.648022 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:57.647980    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 16:59:58.286958 ip-10-0-135-55 kubenswrapper[2569]: I0911 16:59:58.286924    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:00:00.164263 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.164230    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f"]
Sep 11 17:00:00.167914 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.167893    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v"]
Sep 11 17:00:00.168049 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.168030    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f"
Sep 11 17:00:00.170388 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.170367    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 17:00:00.171314 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.171295    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v"
Sep 11 17:00:00.181709 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.181680    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f"]
Sep 11 17:00:00.182010 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.181990    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f"
Sep 11 17:00:00.184136 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.184119    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v"]
Sep 11 17:00:00.187032 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.186623    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h"]
Sep 11 17:00:00.190626 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.190606    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h"
Sep 11 17:00:00.190903 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.190887    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v"
Sep 11 17:00:00.198141 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.198115    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h"]
Sep 11 17:00:00.228124 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.228097    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h"
Sep 11 17:00:00.470293 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.470268    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f"]
Sep 11 17:00:00.472151 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:00:00.472124    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c770ad7_9e94_485d_b71c_6cfe645e7366.slice/crio-722913dbd9e5fd1269644229eee9db09657b6e8b348f6dcaa5be53bee8eaa4af WatchSource:0}: Error finding container 722913dbd9e5fd1269644229eee9db09657b6e8b348f6dcaa5be53bee8eaa4af: Status 404 returned error can't find the container with id 722913dbd9e5fd1269644229eee9db09657b6e8b348f6dcaa5be53bee8eaa4af
Sep 11 17:00:00.478004 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.477935    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v"]
Sep 11 17:00:00.479273 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:00:00.479248    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e23dccc_fd2a_4533_84bf_71569464db65.slice/crio-3cdfa96a1aba6df1177852f51635f0d83e92e569c9e2c4276ac4c8d10368cb97 WatchSource:0}: Error finding container 3cdfa96a1aba6df1177852f51635f0d83e92e569c9e2c4276ac4c8d10368cb97: Status 404 returned error can't find the container with id 3cdfa96a1aba6df1177852f51635f0d83e92e569c9e2c4276ac4c8d10368cb97
Sep 11 17:00:00.521726 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.521699    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h"]
Sep 11 17:00:00.523366 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:00:00.523339    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a57d78a_d225_4821_9a04_b5edf02b86eb.slice/crio-0927dcb887385e76701330759704ddb7578d521cf96938c743f518ee862388f8 WatchSource:0}: Error finding container 0927dcb887385e76701330759704ddb7578d521cf96938c743f518ee862388f8: Status 404 returned error can't find the container with id 0927dcb887385e76701330759704ddb7578d521cf96938c743f518ee862388f8
Sep 11 17:00:00.637512 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.637468    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v" event={"ID":"2e23dccc-fd2a-4533-84bf-71569464db65","Type":"ContainerStarted","Data":"3cdfa96a1aba6df1177852f51635f0d83e92e569c9e2c4276ac4c8d10368cb97"}
Sep 11 17:00:00.638869 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.638838    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f" event={"ID":"5c770ad7-9e94-485d-b71c-6cfe645e7366","Type":"ContainerStarted","Data":"722913dbd9e5fd1269644229eee9db09657b6e8b348f6dcaa5be53bee8eaa4af"}
Sep 11 17:00:00.640202 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:00.640179    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h" event={"ID":"3a57d78a-d225-4821-9a04-b5edf02b86eb","Type":"ContainerStarted","Data":"0927dcb887385e76701330759704ddb7578d521cf96938c743f518ee862388f8"}
Sep 11 17:00:01.648232 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:01.648196    2569 generic.go:356] "Generic (PLEG): container finished" podID="3a57d78a-d225-4821-9a04-b5edf02b86eb" containerID="f5576b066ed43103e31b6b0627a9c35d63729f6ce907a992a6aeb92eba769e22" exitCode=0
Sep 11 17:00:01.648702 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:01.648283    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h" event={"ID":"3a57d78a-d225-4821-9a04-b5edf02b86eb","Type":"ContainerDied","Data":"f5576b066ed43103e31b6b0627a9c35d63729f6ce907a992a6aeb92eba769e22"}
Sep 11 17:00:01.649980 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:01.649953    2569 generic.go:356] "Generic (PLEG): container finished" podID="2e23dccc-fd2a-4533-84bf-71569464db65" containerID="953d02b982e263ba0a14f165ddf422f3e4ebe84e3b119685a608093f60a67f2c" exitCode=0
Sep 11 17:00:01.650121 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:01.650038    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v" event={"ID":"2e23dccc-fd2a-4533-84bf-71569464db65","Type":"ContainerDied","Data":"953d02b982e263ba0a14f165ddf422f3e4ebe84e3b119685a608093f60a67f2c"}
Sep 11 17:00:01.651672 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:01.651651    2569 generic.go:356] "Generic (PLEG): container finished" podID="5c770ad7-9e94-485d-b71c-6cfe645e7366" containerID="35a9798da5a1bea430aef7c29ee784bb2d331f902c70a5561e43176eff403afc" exitCode=0
Sep 11 17:00:01.651766 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:01.651703    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f" event={"ID":"5c770ad7-9e94-485d-b71c-6cfe645e7366","Type":"ContainerDied","Data":"35a9798da5a1bea430aef7c29ee784bb2d331f902c70a5561e43176eff403afc"}
Sep 11 17:00:02.859375 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:02.859337    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f"
Sep 11 17:00:02.862552 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:02.862527    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v"
Sep 11 17:00:02.866399 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:02.866377    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h"
Sep 11 17:00:03.664400 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:03.664368    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v"
Sep 11 17:00:03.664588 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:03.664397    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v" event={"ID":"2e23dccc-fd2a-4533-84bf-71569464db65","Type":"ContainerDied","Data":"3cdfa96a1aba6df1177852f51635f0d83e92e569c9e2c4276ac4c8d10368cb97"}
Sep 11 17:00:03.664588 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:03.664446    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3cdfa96a1aba6df1177852f51635f0d83e92e569c9e2c4276ac4c8d10368cb97"
Sep 11 17:00:03.665850 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:03.665818    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f"
Sep 11 17:00:03.665989 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:03.665856    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f" event={"ID":"5c770ad7-9e94-485d-b71c-6cfe645e7366","Type":"ContainerDied","Data":"722913dbd9e5fd1269644229eee9db09657b6e8b348f6dcaa5be53bee8eaa4af"}
Sep 11 17:00:03.665989 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:03.665901    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="722913dbd9e5fd1269644229eee9db09657b6e8b348f6dcaa5be53bee8eaa4af"
Sep 11 17:00:03.667470 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:03.667437    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h" event={"ID":"3a57d78a-d225-4821-9a04-b5edf02b86eb","Type":"ContainerDied","Data":"0927dcb887385e76701330759704ddb7578d521cf96938c743f518ee862388f8"}
Sep 11 17:00:03.667470 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:03.667457    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h"
Sep 11 17:00:03.667619 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:03.667485    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0927dcb887385e76701330759704ddb7578d521cf96938c743f518ee862388f8"
Sep 11 17:00:04.143428 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.143394    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:04.242485 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.242450    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:04.441391 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.441303    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"]
Sep 11 17:00:04.441966 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.441945    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c770ad7-9e94-485d-b71c-6cfe645e7366" containerName="cleanup"
Sep 11 17:00:04.441966 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.441966    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c770ad7-9e94-485d-b71c-6cfe645e7366" containerName="cleanup"
Sep 11 17:00:04.442154 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.441980    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e23dccc-fd2a-4533-84bf-71569464db65" containerName="cleanup"
Sep 11 17:00:04.442154 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.441986    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e23dccc-fd2a-4533-84bf-71569464db65" containerName="cleanup"
Sep 11 17:00:04.442154 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.441997    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a57d78a-d225-4821-9a04-b5edf02b86eb" containerName="cleanup"
Sep 11 17:00:04.442154 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.442003    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a57d78a-d225-4821-9a04-b5edf02b86eb" containerName="cleanup"
Sep 11 17:00:04.442154 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.442065    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c770ad7-9e94-485d-b71c-6cfe645e7366" containerName="cleanup"
Sep 11 17:00:04.442154 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.442105    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e23dccc-fd2a-4533-84bf-71569464db65" containerName="cleanup"
Sep 11 17:00:04.442154 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.442114    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a57d78a-d225-4821-9a04-b5edf02b86eb" containerName="cleanup"
Sep 11 17:00:04.445329 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.445310    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.447445 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.447422    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-31e12f72-openshift-default-dockercfg-6zqgz\""
Sep 11 17:00:04.460159 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.460122    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"]
Sep 11 17:00:04.585805 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.585771    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-token\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.585805 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.585810    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-envoy\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.586110 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.585840    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4nch\" (UniqueName: \"kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-kube-api-access-b4nch\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.586110 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.585900    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istiod-ca-cert\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.586110 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.585989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-socket\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.586110 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.586088    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-podinfo\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.586334 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.586150    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-certs\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.586334 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.586187    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-credential-socket\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.586334 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.586263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-data\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.686876 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.686838    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-token\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.686876 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.686877    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-envoy\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687191 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.686908    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4nch\" (UniqueName: \"kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-kube-api-access-b4nch\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687191 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.686940    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istiod-ca-cert\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687191 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.686988    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-socket\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687191 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.687063    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-podinfo\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687191 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.687137    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-certs\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687191 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.687174    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-credential-socket\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687494 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.687235    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-data\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687494 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.687407    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-socket\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687596 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.687490    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-certs\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687695 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.687644    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-data\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687766 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.687726    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-credential-socket\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.687811 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.687770    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istiod-ca-cert\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.689836 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.689804    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-envoy\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.690130 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.690112    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-podinfo\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.695641 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.695575    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-token\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.695746 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.695697    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4nch\" (UniqueName: \"kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-kube-api-access-b4nch\") pod \"e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") " pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:04.733681 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.733644    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:04.756031 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:04.756002    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:05.060639 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:00:05.060605    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf49c922c_47e9_422d_8e9b_b6be4ed4d6ec.slice/crio-01911cfd8fb44e355da7331859bd4f1e868f61086b027587b76201cfffa25926 WatchSource:0}: Error finding container 01911cfd8fb44e355da7331859bd4f1e868f61086b027587b76201cfffa25926: Status 404 returned error can't find the container with id 01911cfd8fb44e355da7331859bd4f1e868f61086b027587b76201cfffa25926
Sep 11 17:00:05.063882 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:05.063848    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:00:05.063979 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:05.063920    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:00:05.063979 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:05.063948    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:00:05.065454 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:05.065432    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"]
Sep 11 17:00:05.275250 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:05.275214    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:05.681747 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:05.681715    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" event={"ID":"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec","Type":"ContainerStarted","Data":"4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072"}
Sep 11 17:00:05.681920 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:05.681767    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" event={"ID":"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec","Type":"ContainerStarted","Data":"01911cfd8fb44e355da7331859bd4f1e868f61086b027587b76201cfffa25926"}
Sep 11 17:00:05.726395 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:05.726332    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" podStartSLOduration=1.726315654 podStartE2EDuration="1.726315654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:00:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:00:05.724991282 +0000 UTC m=+1002.132209949" watchObservedRunningTime="2026-09-11 17:00:05.726315654 +0000 UTC m=+1002.133534302"
Sep 11 17:00:05.756206 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:05.756163    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:06.762007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:06.761976    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:07.693059 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:07.693015    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:07.693919 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:07.693900    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:10.080886 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:10.080849    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:10.804781 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:10.804728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:10.898794 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:10.898739    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:11.082402 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:11.082306    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:11.858604 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:11.858567    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:12.902594 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:12.902557    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:13.660043 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:13.660003    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:14.467721 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:14.467685    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:15.222275 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:15.222240    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:15.338500 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:15.338468    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:15.566697 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:15.566604    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:15.884626 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:15.884591    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:15.983290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:15.983237    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:19.980410 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:19.980379    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:00:21.490009 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:21.489969    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:26.981171 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:26.981138    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:37.022213 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:37.022181    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:37.145606 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:37.145573    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:37.806006 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:37.805962    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:38.519308 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:38.519274    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:39.394197 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:39.394160    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:40.074634 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:40.074599    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:40.863461 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:40.863423    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:40.982116 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:40.982065    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:41.590282 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:41.590247    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:41.690469 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:41.690436    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:41.791293 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:41.791260    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:42.120099 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:42.120036    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:42.490848 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:42.490809    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:46.925660 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:46.925627    2569 generic.go:356] "Generic (PLEG): container finished" podID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerID="4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072" exitCode=0
Sep 11 17:00:46.926038 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:46.925692    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" event={"ID":"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec","Type":"ContainerDied","Data":"4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072"}
Sep 11 17:00:46.926174 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:46.926158    2569 scope.go:117] "RemoveContainer" containerID="4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072"
Sep 11 17:00:46.927421 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:46.927390    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:00:46.927513 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:46.927457    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:00:46.927513 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:46.927488    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:00:47.937845 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:47.936981    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" event={"ID":"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec","Type":"ContainerStarted","Data":"c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156"}
Sep 11 17:00:48.706062 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:48.706025    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:48.757018 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:48.756976    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:48.758381 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:48.758353    2569 patch_prober.go:28] interesting pod/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 11 17:00:48.758505 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:48.758407    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 11 17:00:48.766861 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:48.766834    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:49.756786 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:49.756741    2569 patch_prober.go:28] interesting pod/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 11 17:00:49.757319 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:49.756835    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 11 17:00:49.943010 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:49.942946    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:50.650218 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.650091    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:50.744172 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.744130    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"]
Sep 11 17:00:50.748141 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.748108    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"
Sep 11 17:00:50.751255 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.751225    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:00:50.752166 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.752143    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 17:00:50.756718 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.756688    2569 patch_prober.go:28] interesting pod/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 11 17:00:50.756986 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.756956    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 11 17:00:50.761351 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.761324    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"]
Sep 11 17:00:50.816632 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.816586    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw\" (UID: \"eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"
Sep 11 17:00:50.917489 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.917391    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw\" (UID: \"eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"
Sep 11 17:00:50.918091 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.918051    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw\" (UID: \"eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"
Sep 11 17:00:50.979366 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:50.979313    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:51.064349 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:51.064301    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"
Sep 11 17:00:51.339836 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:51.339721    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"]
Sep 11 17:00:51.348753 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:00:51.348712    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb9f86fe_96a0_4c53_ba41_cc92f0eaf05f.slice/crio-6b74eb2b1d45e988d80a7d76f388d78d1af86d90fb298ec8fb87bb4a9af0d675 WatchSource:0}: Error finding container 6b74eb2b1d45e988d80a7d76f388d78d1af86d90fb298ec8fb87bb4a9af0d675: Status 404 returned error can't find the container with id 6b74eb2b1d45e988d80a7d76f388d78d1af86d90fb298ec8fb87bb4a9af0d675
Sep 11 17:00:51.756435 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:51.756397    2569 patch_prober.go:28] interesting pod/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 11 17:00:51.757137 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:51.756711    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 11 17:00:51.796064 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:51.796019    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:51.969450 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:51.969397    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw" event={"ID":"eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f","Type":"ContainerStarted","Data":"7e52eb976984a18e97af4e724a689a0b93fc17c5b65fcec9c9adf007e9e3d4a1"}
Sep 11 17:00:51.969657 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:51.969512    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw" event={"ID":"eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f","Type":"ContainerStarted","Data":"6b74eb2b1d45e988d80a7d76f388d78d1af86d90fb298ec8fb87bb4a9af0d675"}
Sep 11 17:00:51.994152 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:51.994067    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw" podStartSLOduration=1.994042713 podStartE2EDuration="1.994042713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:00:51.990705127 +0000 UTC m=+1048.397923821" watchObservedRunningTime="2026-09-11 17:00:51.994042713 +0000 UTC m=+1048.401261361"
Sep 11 17:00:52.685410 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:52.685373    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:52.693694 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:52.693659    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:52.758178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:52.758138    2569 patch_prober.go:28] interesting pod/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 11 17:00:52.758694 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:52.758207    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 11 17:00:52.977824 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:52.977733    2569 generic.go:356] "Generic (PLEG): container finished" podID="eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f" containerID="7e52eb976984a18e97af4e724a689a0b93fc17c5b65fcec9c9adf007e9e3d4a1" exitCode=0
Sep 11 17:00:52.977970 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:52.977823    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw" event={"ID":"eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f","Type":"ContainerDied","Data":"7e52eb976984a18e97af4e724a689a0b93fc17c5b65fcec9c9adf007e9e3d4a1"}
Sep 11 17:00:53.575976 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:53.575944    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:53.760645 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:53.760606    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:53.761606 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:53.761589    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:00:54.129213 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:54.129184    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"
Sep 11 17:00:54.275804 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:54.275711    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f-maas-api-service-ca\") pod \"eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f\" (UID: \"eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f\") "
Sep 11 17:00:54.276146 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:54.276120    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f" (UID: "eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:00:54.351505 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:54.351465    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:54.377655 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:54.377626    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:00:54.994804 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:54.994775    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"
Sep 11 17:00:54.994804 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:54.994789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw" event={"ID":"eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f","Type":"ContainerDied","Data":"6b74eb2b1d45e988d80a7d76f388d78d1af86d90fb298ec8fb87bb4a9af0d675"}
Sep 11 17:00:54.995278 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:54.994846    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b74eb2b1d45e988d80a7d76f388d78d1af86d90fb298ec8fb87bb4a9af0d675"
Sep 11 17:00:55.051618 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:55.051581    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:55.779270 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:55.779235    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:00:55.789907 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:55.789869    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:56.492025 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:56.491990    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:59.276149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:59.276111    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:59.905907 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:59.905866    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:00:59.978968 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:00:59.978934    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:00.078843 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:00.078810    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:00.339393 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:00.339282    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:00.483406 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:00.483373    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:00.584255 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:00.584224    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:01.158550 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:01.158521    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:01.282523 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:01.282466    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:01.382994 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:01.382953    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:01.476567 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:01.476491    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:01.519330 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:01.519295    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"]
Sep 11 17:01:01.521515 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:01.521493    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2cceea-bxmkw"]
Sep 11 17:01:01.708392 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:01.708356    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:01.907942 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:01.907913    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:02.206356 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:02.206272    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f" path="/var/lib/kubelet/pods/eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f/volumes"
Sep 11 17:01:02.560193 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:02.560106    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:03.459642 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:03.459608    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:04.257703 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:04.257662    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:04.980217 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:04.980183    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:05.675697 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:05.675666    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:06.809655 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:06.809625    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:06.913500 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:06.913465    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:07.070412 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:07.070331    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"]
Sep 11 17:01:07.070627 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:07.070596    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy" containerID="cri-o://c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156" gracePeriod=30
Sep 11 17:01:07.072779 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:07.072752    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:07.434200 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:07.434164    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:07.536275 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:07.536244    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:07.646449 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:07.646415    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:08.464487 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:08.464453    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:08.564499 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:08.564458    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:08.718939 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:08.718854    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"]
Sep 11 17:01:08.719191 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:08.719161    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy" containerID="cri-o://8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3" gracePeriod=30
Sep 11 17:01:08.763850 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:08.763808    2569 patch_prober.go:28] interesting pod/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 17:01:08.764016 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:08.763868    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:01:08.771749 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:08.771720    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:09.203447 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:09.203414    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:09.548561 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:09.548475    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:09.664901 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:09.664867    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:09.756037 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:09.756009    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:12.332799 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.332729    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:01:12.375471 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.375437    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-token\") pod \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") "
Sep 11 17:01:12.375656 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.375493    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-credential-socket\") pod \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") "
Sep 11 17:01:12.375656 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.375551    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-podinfo\") pod \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") "
Sep 11 17:01:12.375656 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.375614    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-certs\") pod \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") "
Sep 11 17:01:12.375656 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.375653    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-data\") pod \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") "
Sep 11 17:01:12.376414 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.375683    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-socket\") pod \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") "
Sep 11 17:01:12.376414 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.375744    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-envoy\") pod \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") "
Sep 11 17:01:12.376414 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.375770    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istiod-ca-cert\") pod \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") "
Sep 11 17:01:12.376414 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.375800    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4nch\" (UniqueName: \"kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-kube-api-access-b4nch\") pod \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\" (UID: \"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec\") "
Sep 11 17:01:12.376414 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.375924    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" (UID: "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:01:12.376414 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.376008    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" (UID: "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:01:12.376414 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.376381    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:12.376414 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.376402    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:12.376835 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.376443    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-data" (OuterVolumeSpecName: "istio-data") pod "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" (UID: "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:01:12.376835 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.376459    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" (UID: "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:01:12.376835 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.376462    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" (UID: "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:01:12.378339 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.378286    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" (UID: "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:01:12.378898 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.378866    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-kube-api-access-b4nch" (OuterVolumeSpecName: "kube-api-access-b4nch") pod "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" (UID: "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec"). InnerVolumeSpecName "kube-api-access-b4nch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:01:12.379010 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.378964    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-token" (OuterVolumeSpecName: "istio-token") pod "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" (UID: "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:01:12.379010 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.378983    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" (UID: "f49c922c-47e9-422d-8e9b-b6be4ed4d6ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:01:12.477791 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.477739    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:12.477791 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.477785    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:12.477791 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.477796    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4nch\" (UniqueName: \"kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-kube-api-access-b4nch\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:12.477791 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.477806    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:12.478070 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.477815    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:12.478070 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.477822    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:12.478070 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:12.477830    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:13.103603 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.103561    2569 generic.go:356] "Generic (PLEG): container finished" podID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerID="c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156" exitCode=0
Sep 11 17:01:13.103795 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.103614    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"
Sep 11 17:01:13.103795 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.103620    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" event={"ID":"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec","Type":"ContainerDied","Data":"c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156"}
Sep 11 17:01:13.103795 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.103675    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7" event={"ID":"f49c922c-47e9-422d-8e9b-b6be4ed4d6ec","Type":"ContainerDied","Data":"01911cfd8fb44e355da7331859bd4f1e868f61086b027587b76201cfffa25926"}
Sep 11 17:01:13.103795 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.103704    2569 scope.go:117] "RemoveContainer" containerID="c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156"
Sep 11 17:01:13.114149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.114125    2569 scope.go:117] "RemoveContainer" containerID="4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072"
Sep 11 17:01:13.123249 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.123228    2569 scope.go:117] "RemoveContainer" containerID="c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156"
Sep 11 17:01:13.123558 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:01:13.123534    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156\": container with ID starting with c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156 not found: ID does not exist" containerID="c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156"
Sep 11 17:01:13.123635 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.123571    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156"} err="failed to get container status \"c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156\": rpc error: code = NotFound desc = could not find container \"c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156\": container with ID starting with c114fc77f3f7718346687f9718a8a8238268246f75355946c75aecb15b2a1156 not found: ID does not exist"
Sep 11 17:01:13.123635 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.123598    2569 scope.go:117] "RemoveContainer" containerID="4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072"
Sep 11 17:01:13.123915 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:01:13.123896    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072\": container with ID starting with 4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072 not found: ID does not exist" containerID="4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072"
Sep 11 17:01:13.123956 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.123921    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072"} err="failed to get container status \"4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072\": rpc error: code = NotFound desc = could not find container \"4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072\": container with ID starting with 4482b2bdca9064d6d0da01d821ddd6ab6c412fc6d1b8938e78aa3cc38f0fe072 not found: ID does not exist"
Sep 11 17:01:13.137584 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.137544    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"]
Sep 11 17:01:13.142598 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.142563    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-31e12f72-openshift-default-f9b7cbb8-vr8t7"]
Sep 11 17:01:13.212005 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.211975    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:13.306550 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.306517    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:13.447336 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.447296    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:13.984391 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:13.984326    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 17:01:14.062887 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.062852    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:14.093905 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.093865    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-podinfo\") pod \"cfe259e3-e59a-4c41-a782-b1941356f8b9\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") "
Sep 11 17:01:14.093905 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.093914    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-socket\") pod \"cfe259e3-e59a-4c41-a782-b1941356f8b9\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") "
Sep 11 17:01:14.094179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.093955    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-certs\") pod \"cfe259e3-e59a-4c41-a782-b1941356f8b9\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") "
Sep 11 17:01:14.094179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.093986    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-credential-socket\") pod \"cfe259e3-e59a-4c41-a782-b1941356f8b9\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") "
Sep 11 17:01:14.094179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094015    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-envoy\") pod \"cfe259e3-e59a-4c41-a782-b1941356f8b9\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") "
Sep 11 17:01:14.094179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094055    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-data\") pod \"cfe259e3-e59a-4c41-a782-b1941356f8b9\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") "
Sep 11 17:01:14.094179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094111    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-token\") pod \"cfe259e3-e59a-4c41-a782-b1941356f8b9\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") "
Sep 11 17:01:14.094179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094141    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfe259e3-e59a-4c41-a782-b1941356f8b9-istiod-ca-cert\") pod \"cfe259e3-e59a-4c41-a782-b1941356f8b9\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") "
Sep 11 17:01:14.094552 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094264    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8k4rp\" (UniqueName: \"kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-kube-api-access-8k4rp\") pod \"cfe259e3-e59a-4c41-a782-b1941356f8b9\" (UID: \"cfe259e3-e59a-4c41-a782-b1941356f8b9\") "
Sep 11 17:01:14.094552 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094335    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cfe259e3-e59a-4c41-a782-b1941356f8b9" (UID: "cfe259e3-e59a-4c41-a782-b1941356f8b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:01:14.094552 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094363    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cfe259e3-e59a-4c41-a782-b1941356f8b9" (UID: "cfe259e3-e59a-4c41-a782-b1941356f8b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:01:14.094552 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094421    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "cfe259e3-e59a-4c41-a782-b1941356f8b9" (UID: "cfe259e3-e59a-4c41-a782-b1941356f8b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:01:14.094552 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094346    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cfe259e3-e59a-4c41-a782-b1941356f8b9" (UID: "cfe259e3-e59a-4c41-a782-b1941356f8b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:01:14.094803 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094700    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfe259e3-e59a-4c41-a782-b1941356f8b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cfe259e3-e59a-4c41-a782-b1941356f8b9" (UID: "cfe259e3-e59a-4c41-a782-b1941356f8b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:01:14.094803 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094765    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:14.094803 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094789    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:14.094931 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094804    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:14.094931 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.094818    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:14.096355 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.096327    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cfe259e3-e59a-4c41-a782-b1941356f8b9" (UID: "cfe259e3-e59a-4c41-a782-b1941356f8b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:01:14.096355 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.096343    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cfe259e3-e59a-4c41-a782-b1941356f8b9" (UID: "cfe259e3-e59a-4c41-a782-b1941356f8b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:01:14.096829 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.096801    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-kube-api-access-8k4rp" (OuterVolumeSpecName: "kube-api-access-8k4rp") pod "cfe259e3-e59a-4c41-a782-b1941356f8b9" (UID: "cfe259e3-e59a-4c41-a782-b1941356f8b9"). InnerVolumeSpecName "kube-api-access-8k4rp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:01:14.096927 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.096825    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "cfe259e3-e59a-4c41-a782-b1941356f8b9" (UID: "cfe259e3-e59a-4c41-a782-b1941356f8b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:01:14.111168 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.111136    2569 generic.go:356] "Generic (PLEG): container finished" podID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerID="8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3" exitCode=0
Sep 11 17:01:14.111344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.111185    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"
Sep 11 17:01:14.111344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.111185    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" event={"ID":"cfe259e3-e59a-4c41-a782-b1941356f8b9","Type":"ContainerDied","Data":"8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3"}
Sep 11 17:01:14.111344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.111269    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq" event={"ID":"cfe259e3-e59a-4c41-a782-b1941356f8b9","Type":"ContainerDied","Data":"d02734d2bc7ebcf9e0d4cedbb1f2e9e74b225240fe009e8ffa006d6978507c30"}
Sep 11 17:01:14.111344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.111293    2569 scope.go:117] "RemoveContainer" containerID="8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3"
Sep 11 17:01:14.121922 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.121893    2569 scope.go:117] "RemoveContainer" containerID="dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f"
Sep 11 17:01:14.131623 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.131594    2569 scope.go:117] "RemoveContainer" containerID="8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3"
Sep 11 17:01:14.131968 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:01:14.131947    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3\": container with ID starting with 8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3 not found: ID does not exist" containerID="8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3"
Sep 11 17:01:14.132049 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.131977    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3"} err="failed to get container status \"8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3\": rpc error: code = NotFound desc = could not find container \"8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3\": container with ID starting with 8619485f738c20de6cbfb3f3fb61436c23ba8bd1454f7b1d43ff2e1550d196b3 not found: ID does not exist"
Sep 11 17:01:14.132049 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.131996    2569 scope.go:117] "RemoveContainer" containerID="dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f"
Sep 11 17:01:14.132332 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:01:14.132314    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f\": container with ID starting with dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f not found: ID does not exist" containerID="dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f"
Sep 11 17:01:14.132383 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.132339    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f"} err="failed to get container status \"dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f\": rpc error: code = NotFound desc = could not find container \"dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f\": container with ID starting with dc2ae107ae9eca5a50ccb1857c019d6fb9fb08eff3c40764c10de8590c8aa26f not found: ID does not exist"
Sep 11 17:01:14.133703 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.133680    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"]
Sep 11 17:01:14.136256 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.136234    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:14.137658 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.137638    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-2cceea-openshift-default-5df55b44fb-s8vgq"]
Sep 11 17:01:14.195734 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.195696    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8k4rp\" (UniqueName: \"kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-kube-api-access-8k4rp\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:14.195734 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.195726    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:14.195734 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.195739    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:14.195977 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.195747    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfe259e3-e59a-4c41-a782-b1941356f8b9-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:14.195977 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.195755    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfe259e3-e59a-4c41-a782-b1941356f8b9-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:01:14.206399 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.206359    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" path="/var/lib/kubelet/pods/cfe259e3-e59a-4c41-a782-b1941356f8b9/volumes"
Sep 11 17:01:14.206929 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.206906    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" path="/var/lib/kubelet/pods/f49c922c-47e9-422d-8e9b-b6be4ed4d6ec/volumes"
Sep 11 17:01:14.218572 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.218544    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:14.322813 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.322737    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:14.417132 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:14.417065    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:20.718149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:20.718045    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:20.780506 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:20.780471    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:21.735304 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:21.735270    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:22.049301 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049202    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"]
Sep 11 17:01:22.049677 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049660    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy"
Sep 11 17:01:22.049725 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049679    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy"
Sep 11 17:01:22.049725 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049693    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f" containerName="revoke-keys"
Sep 11 17:01:22.049725 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049702    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f" containerName="revoke-keys"
Sep 11 17:01:22.049725 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049717    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy"
Sep 11 17:01:22.049725 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049724    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy"
Sep 11 17:01:22.049881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049732    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy"
Sep 11 17:01:22.049881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049737    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy"
Sep 11 17:01:22.049881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049755    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy"
Sep 11 17:01:22.049881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049761    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy"
Sep 11 17:01:22.049881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049841    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy"
Sep 11 17:01:22.049881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049853    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb9f86fe-96a0-4c53-ba41-cc92f0eaf05f" containerName="revoke-keys"
Sep 11 17:01:22.049881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049864    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy"
Sep 11 17:01:22.049881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049870    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfe259e3-e59a-4c41-a782-b1941356f8b9" containerName="istio-proxy"
Sep 11 17:01:22.049881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.049877    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f49c922c-47e9-422d-8e9b-b6be4ed4d6ec" containerName="istio-proxy"
Sep 11 17:01:22.052148 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.052129    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.054464 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.054448    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-6b6653-openshift-default-dockercfg-lwjlv\""
Sep 11 17:01:22.065451 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.065422    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"]
Sep 11 17:01:22.072036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.072012    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:22.183974 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.183906    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-podinfo\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.184242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.184219    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-socket\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.184313 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.184259    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brtbt\" (UniqueName: \"kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-kube-api-access-brtbt\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.184372 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.184309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-token\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.184372 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.184351    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istiod-ca-cert\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.184468 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.184409    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-data\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.184575 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.184538    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-credential-socket\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.184700 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.184672    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-certs\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.184788 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.184770    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-envoy\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.285693 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.285654    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-data\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.285897 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.285706    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-credential-socket\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.285897 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.285751    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-certs\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.285897 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.285783    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-envoy\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.285897 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.285812    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-podinfo\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.285897 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.285883    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-socket\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.286221 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.285909    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brtbt\" (UniqueName: \"kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-kube-api-access-brtbt\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.286221 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.285963    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-token\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.286221 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.286003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istiod-ca-cert\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.286221 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.286192    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-data\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.286420 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.286350    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-certs\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.286420 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.286400    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-credential-socket\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.286580 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.286557    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-socket\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.286780 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.286765    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istiod-ca-cert\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.288330 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.288303    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-envoy\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.288549 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.288526    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-podinfo\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.294233 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.294203    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-token\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.294511 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.294490    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brtbt\" (UniqueName: \"kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-kube-api-access-brtbt\") pod \"e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") " pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.362723 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.362630    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:22.640429 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.640400    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"]
Sep 11 17:01:22.642205 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:01:22.642179    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70deb7d5_a0b5_4ce1_ac24_e0c19619c8cb.slice/crio-95d99c963652f8f6c9279489f04726fc5badbf12ad81343b84ff64e55fafff83 WatchSource:0}: Error finding container 95d99c963652f8f6c9279489f04726fc5badbf12ad81343b84ff64e55fafff83: Status 404 returned error can't find the container with id 95d99c963652f8f6c9279489f04726fc5badbf12ad81343b84ff64e55fafff83
Sep 11 17:01:22.644500 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.644465    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:01:22.644606 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.644543    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:01:22.644606 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.644587    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:01:22.711946 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:22.711913    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:23.176313 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:23.176272    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" event={"ID":"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb","Type":"ContainerStarted","Data":"d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1"}
Sep 11 17:01:23.176780 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:23.176367    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" event={"ID":"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb","Type":"ContainerStarted","Data":"95d99c963652f8f6c9279489f04726fc5badbf12ad81343b84ff64e55fafff83"}
Sep 11 17:01:23.196564 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:23.196512    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" podStartSLOduration=1.196496176 podStartE2EDuration="1.196496176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:01:23.195285134 +0000 UTC m=+1079.602503782" watchObservedRunningTime="2026-09-11 17:01:23.196496176 +0000 UTC m=+1079.603714826"
Sep 11 17:01:23.363655 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:23.363610    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:23.368719 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:23.368685    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:24.181873 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:24.181841    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:24.182808 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:24.182786    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:01:25.084206 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:25.084175    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:25.194722 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:25.194686    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:25.285458 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:25.285386    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:25.970813 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:25.970778    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:26.783948 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:26.783903    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:27.461390 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:27.461349    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:28.268486 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:28.268441    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:28.362242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:28.362209    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:28.587096 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:28.586988    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:01:28.626010 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:28.625948    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:28.758835 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:28.758804    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:28.860082 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:28.859988    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:38.282985 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:38.282954    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:50.312967 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.312928    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:50.597926 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.597821    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:50.707943 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.707901    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:50.904226 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.904191    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"]
Sep 11 17:01:50.910297 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.910270    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:50.912672 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.912650    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-bff65f-openshift-default-dockercfg-b2ww7\""
Sep 11 17:01:50.919649 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.919619    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"]
Sep 11 17:01:50.957091 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.957025    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:50.993151 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.993107    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-credential-socket\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:50.993357 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.993174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-certs\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:50.993357 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.993210    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-token\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:50.993357 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.993243    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-podinfo\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:50.993357 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.993334    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-socket\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:50.993527 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.993375    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istiod-ca-cert\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:50.993527 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.993403    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-data\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:50.993527 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.993466    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-envoy\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:50.993527 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:50.993483    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plz48\" (UniqueName: \"kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-kube-api-access-plz48\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.095142 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.095099    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-credential-socket\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.095356 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.095164    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-certs\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.095356 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.095208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-token\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.095356 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.095250    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-podinfo\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.095356 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.095319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-socket\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.095356 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.095347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istiod-ca-cert\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.095622 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.095384    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-data\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.095622 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.095470    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-envoy\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.095622 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.095496    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plz48\" (UniqueName: \"kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-kube-api-access-plz48\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.096370 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.096321    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-credential-socket\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.096665 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.096612    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-certs\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.097717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.097671    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-data\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.098505 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.098454    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istiod-ca-cert\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.098940 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.098916    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-socket\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.100442 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.100418    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-podinfo\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.101196 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.101178    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-envoy\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.105365 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.105333    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-token\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.106342 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.106318    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plz48\" (UniqueName: \"kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-kube-api-access-plz48\") pod \"e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") " pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.124841 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.124813    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:51.222802 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.222700    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:51.489343 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.489233    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:51.525914 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.525890    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"]
Sep 11 17:01:51.527843 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:01:51.527813    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e34fc3_2c43_4c18_9445_c344c60ecc86.slice/crio-e9278af24f57f297686856fad4e8c9d59fde60ff16630fe9ee705ffe3baa4d05 WatchSource:0}: Error finding container e9278af24f57f297686856fad4e8c9d59fde60ff16630fe9ee705ffe3baa4d05: Status 404 returned error can't find the container with id e9278af24f57f297686856fad4e8c9d59fde60ff16630fe9ee705ffe3baa4d05
Sep 11 17:01:51.539102 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.530909    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:01:51.539102 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.530999    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:01:51.539102 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:51.531046    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:01:52.279419 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:52.279383    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:52.369016 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:52.368978    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" event={"ID":"f9e34fc3-2c43-4c18-9445-c344c60ecc86","Type":"ContainerStarted","Data":"d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b"}
Sep 11 17:01:52.369242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:52.369033    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" event={"ID":"f9e34fc3-2c43-4c18-9445-c344c60ecc86","Type":"ContainerStarted","Data":"e9278af24f57f297686856fad4e8c9d59fde60ff16630fe9ee705ffe3baa4d05"}
Sep 11 17:01:52.405917 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:52.405856    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" podStartSLOduration=2.405840129 podStartE2EDuration="2.405840129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:01:52.401721484 +0000 UTC m=+1108.808940151" watchObservedRunningTime="2026-09-11 17:01:52.405840129 +0000 UTC m=+1108.813058777"
Sep 11 17:01:53.223191 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:53.223151    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:53.228639 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:53.228610    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:53.374998 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:53.374967    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:53.376151 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:53.376129    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:01:53.507305 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:53.507216    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:54.375679 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:54.375645    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:54.477289 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:54.477251    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:54.587309 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:54.587273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:55.379138 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:55.379100    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:56.392714 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:56.392677    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:56.493330 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:56.493293    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:56.596484 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:56.596452    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:56.862536 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:56.862377    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:56.943913 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:56.943875    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:57.293653 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:57.293609    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:58.137946 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:58.137908    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:58.830243 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:58.830216    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:01:59.414647 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:59.414613    2569 generic.go:356] "Generic (PLEG): container finished" podID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerID="d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1" exitCode=0
Sep 11 17:01:59.415062 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:59.414687    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" event={"ID":"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb","Type":"ContainerDied","Data":"d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1"}
Sep 11 17:01:59.415181 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:59.415166    2569 scope.go:117] "RemoveContainer" containerID="d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1"
Sep 11 17:01:59.416540 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:59.416517    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:01:59.416611 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:59.416588    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:01:59.416648 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:59.416622    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:01:59.603390 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:01:59.603346    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:00.422331 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:00.422292    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" event={"ID":"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb","Type":"ContainerStarted","Data":"7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af"}
Sep 11 17:02:00.471336 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:00.471303    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:00.794154 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:00.794053    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:02:01.363554 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:01.363524    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:02:01.363792 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:01.363770    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 11 17:02:01.363851 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:01.363824    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 11 17:02:02.363598 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:02.363513    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 11 17:02:02.363598 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:02.363578    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 11 17:02:03.363929 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:03.363893    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 11 17:02:03.364464 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:03.363956    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 11 17:02:04.362978 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:04.362942    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 11 17:02:04.363221 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:04.363018    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 11 17:02:05.364010 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:05.363973    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 11 17:02:05.364444 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:05.364044    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 11 17:02:06.367466 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:06.367438    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:02:06.367869 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:06.367798    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:02:06.368552 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:06.368529    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:02:14.255468 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:14.255413    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:14.318017 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:14.317980    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:17.280710 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.280672    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"]
Sep 11 17:02:17.284781 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.284755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.287244 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.287217    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2dc95c\"/\"e2e-conflict-a-44a0e379-kserve-self-signed-certs\""
Sep 11 17:02:17.298762 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.298726    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"]
Sep 11 17:02:17.451661 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.451612    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05bd90b6-2aac-4a39-929c-28869019a323-tls-certs\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.451878 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.451681    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-model-cache\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.451878 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.451764    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-dshm\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.451878 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.451805    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-home\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.451878 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.451830    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjq89\" (UniqueName: \"kubernetes.io/projected/05bd90b6-2aac-4a39-929c-28869019a323-kube-api-access-rjq89\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.452143 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.451904    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-tmp-dir\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.552764 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.552627    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-home\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.552764 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.552708    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjq89\" (UniqueName: \"kubernetes.io/projected/05bd90b6-2aac-4a39-929c-28869019a323-kube-api-access-rjq89\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.553006 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.552791    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-tmp-dir\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.553006 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.552909    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05bd90b6-2aac-4a39-929c-28869019a323-tls-certs\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.553006 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.552946    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-model-cache\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.553209 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.553028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-dshm\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.553209 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.553160    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-home\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.553877 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.553820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-model-cache\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.553877 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.553845    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-tmp-dir\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.557752 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.557697    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05bd90b6-2aac-4a39-929c-28869019a323-tls-certs\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.560742 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.560711    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-dshm\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.562613 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.562580    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjq89\" (UniqueName: \"kubernetes.io/projected/05bd90b6-2aac-4a39-929c-28869019a323-kube-api-access-rjq89\") pod \"e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") " pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.594654 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.594614    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:17.898944 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:17.898914    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"]
Sep 11 17:02:17.901023 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:02:17.900994    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05bd90b6_2aac_4a39_929c_28869019a323.slice/crio-4503276f7e38db4ee594bfdb54f1f2d55fcaf4dd06da759debb15eca81ce43a1 WatchSource:0}: Error finding container 4503276f7e38db4ee594bfdb54f1f2d55fcaf4dd06da759debb15eca81ce43a1: Status 404 returned error can't find the container with id 4503276f7e38db4ee594bfdb54f1f2d55fcaf4dd06da759debb15eca81ce43a1
Sep 11 17:02:18.541948 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:18.541901    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7" event={"ID":"05bd90b6-2aac-4a39-929c-28869019a323","Type":"ContainerStarted","Data":"3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970"}
Sep 11 17:02:18.541948 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:18.541951    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7" event={"ID":"05bd90b6-2aac-4a39-929c-28869019a323","Type":"ContainerStarted","Data":"4503276f7e38db4ee594bfdb54f1f2d55fcaf4dd06da759debb15eca81ce43a1"}
Sep 11 17:02:18.561640 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:18.561568    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7" podStartSLOduration=1.561541308 podStartE2EDuration="1.561541308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:02:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:02:18.558258961 +0000 UTC m=+1134.965477608" watchObservedRunningTime="2026-09-11 17:02:18.561541308 +0000 UTC m=+1134.968759960"
Sep 11 17:02:19.019533 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:19.019483    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:19.077441 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:19.077408    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:19.142849 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:19.142807    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:19.400856 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:19.400824    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:19.987746 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:19.987714    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:20.904520 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:20.904475    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:21.564288 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:21.564190    2569 generic.go:356] "Generic (PLEG): container finished" podID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerID="d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b" exitCode=0
Sep 11 17:02:21.564690 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:21.564269    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" event={"ID":"f9e34fc3-2c43-4c18-9445-c344c60ecc86","Type":"ContainerDied","Data":"d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b"}
Sep 11 17:02:21.564934 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:21.564919    2569 scope.go:117] "RemoveContainer" containerID="d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b"
Sep 11 17:02:21.566379 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:21.566355    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:02:21.566449 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:21.566422    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:02:21.566486 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:21.566453    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:02:21.647893 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:21.647858    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:21.692652 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:21.692616    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:02:22.503034 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:22.503002    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:22.578106 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:22.577825    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" event={"ID":"f9e34fc3-2c43-4c18-9445-c344c60ecc86","Type":"ContainerStarted","Data":"2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad"}
Sep 11 17:02:23.222919 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:23.222886    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:02:23.224025 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:23.223998    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 11 17:02:23.224171 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:23.224046    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 11 17:02:23.319817 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:23.319780    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:23.375751 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:23.375708    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:02:24.145506 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:24.145456    2569 scope.go:117] "RemoveContainer" containerID="ce4a61b39e0421bb46b2a35e3fd119c11b20b12d29a1af697a902790cf9c952a"
Sep 11 17:02:24.223848 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:24.223812    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 11 17:02:24.224042 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:24.223878    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 11 17:02:25.223911 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:25.223868    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 11 17:02:25.224355 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:25.223931    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 11 17:02:26.223955 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:26.223919    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 11 17:02:26.224363 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:26.223991    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 11 17:02:27.223875 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:27.223838    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 11 17:02:27.224050 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:27.223905    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 11 17:02:27.595488 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:27.595407    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:27.595700 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:27.595671    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:27.608263 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:27.608236    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:27.621773 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:27.621747    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:02:28.227221 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:28.227188    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:02:28.228239 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:28.228215    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:02:28.631152 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:28.631123    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:28.915872 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:28.915783    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"]
Sep 11 17:02:28.919549 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:28.919527    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:28.922192 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:28.922172    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6d422395-openshift-default-dockercfg-j4qhv\""
Sep 11 17:02:28.932173 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:28.932152    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"]
Sep 11 17:02:28.952149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:28.952117    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:29.000445 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.000409    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-socket\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.000445 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.000444    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istiod-ca-cert\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.000652 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.000470    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-podinfo\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.000652 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.000515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-envoy\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.000652 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.000593    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-token\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.000652 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.000638    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-data\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.000787 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.000670    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm6pv\" (UniqueName: \"kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-kube-api-access-wm6pv\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.000787 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.000695    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-certs\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.000787 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.000723    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-credential-socket\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.014627 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.014599    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:29.101972 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.101935    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-data\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102207 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.101996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm6pv\" (UniqueName: \"kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-kube-api-access-wm6pv\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102207 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102029    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-certs\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102207 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-credential-socket\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102378 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-socket\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102378 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istiod-ca-cert\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102378 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102273    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-podinfo\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102378 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102307    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-envoy\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102599 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102377    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-token\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102599 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102413    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-data\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102599 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102450    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-certs\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102599 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102487    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-credential-socket\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102599 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102553    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-socket\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.102998 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.102978    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istiod-ca-cert\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.104715 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.104689    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-envoy\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.104881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.104862    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-podinfo\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.111393 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.111371    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-token\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.111735 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.111711    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm6pv\" (UniqueName: \"kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-kube-api-access-wm6pv\") pod \"e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") " pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.230602 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.230493    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:29.507244 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.507216    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"]
Sep 11 17:02:29.508939 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:02:29.508915    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded4d0c50_8ffc_469f_8c86_2381dd69d887.slice/crio-3ca744272c5a97b7d6f376336b2a4c16f2dccf0d40b9ef7a3cc63aa534450fa4 WatchSource:0}: Error finding container 3ca744272c5a97b7d6f376336b2a4c16f2dccf0d40b9ef7a3cc63aa534450fa4: Status 404 returned error can't find the container with id 3ca744272c5a97b7d6f376336b2a4c16f2dccf0d40b9ef7a3cc63aa534450fa4
Sep 11 17:02:29.511638 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.511597    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:02:29.511715 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.511692    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:02:29.511757 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.511741    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:02:29.625094 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.625024    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" event={"ID":"ed4d0c50-8ffc-469f-8c86-2381dd69d887","Type":"ContainerStarted","Data":"74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da"}
Sep 11 17:02:29.625266 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.625164    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" event={"ID":"ed4d0c50-8ffc-469f-8c86-2381dd69d887","Type":"ContainerStarted","Data":"3ca744272c5a97b7d6f376336b2a4c16f2dccf0d40b9ef7a3cc63aa534450fa4"}
Sep 11 17:02:29.647741 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:29.647689    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" podStartSLOduration=1.647672219 podStartE2EDuration="1.647672219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:02:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:02:29.646114169 +0000 UTC m=+1146.053332833" watchObservedRunningTime="2026-09-11 17:02:29.647672219 +0000 UTC m=+1146.054890864"
Sep 11 17:02:30.231118 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:30.231066    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:30.235864 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:30.235831    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:30.631545 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:30.631509    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:30.632418 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:30.632395    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:02:31.904910 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:31.904874    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:32.617799 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:32.617773    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:32.708853 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:32.708819    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:32.872593 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:32.872506    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:33.780886 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.780855    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"]
Sep 11 17:02:33.783600 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.783583    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.788678 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.788648    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-9459f0\"/\"noten-model-9459f0-kserve-self-signed-certs\""
Sep 11 17:02:33.788678 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.788672    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-9459f0\"/\"kube-root-ca.crt\""
Sep 11 17:02:33.788883 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.788648    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-9459f0\"/\"openshift-service-ca.crt\""
Sep 11 17:02:33.788883 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.788648    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-9459f0\"/\"default-dockercfg-fclvz\""
Sep 11 17:02:33.797047 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.797015    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"]
Sep 11 17:02:33.817788 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.817756    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:33.855013 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.854981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-dshm\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.855194 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.855049    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmjpz\" (UniqueName: \"kubernetes.io/projected/18e1c6ee-fd31-4311-af83-92d8d734e8bc-kube-api-access-fmjpz\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.855194 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.855105    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tls-certs\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.855194 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.855153    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tmp-dir\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.855194 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.855175    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-home\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.855350 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.855243    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-model-cache\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.937068 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.937027    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:33.956485 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.956450    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tmp-dir\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.956599 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.956505    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-home\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.956599 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.956553    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-model-cache\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.956701 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.956651    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-dshm\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.956776 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.956753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmjpz\" (UniqueName: \"kubernetes.io/projected/18e1c6ee-fd31-4311-af83-92d8d734e8bc-kube-api-access-fmjpz\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.956836 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.956799    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tls-certs\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.956934 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.956915    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tmp-dir\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.957211 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.957181    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-home\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.957349 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.957301    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-model-cache\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.959172 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.959149    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-dshm\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.959409 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.959393    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tls-certs\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:33.967260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:33.967238    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmjpz\" (UniqueName: \"kubernetes.io/projected/18e1c6ee-fd31-4311-af83-92d8d734e8bc-kube-api-access-fmjpz\") pod \"noten-model-9459f0-kserve-6d987647-48z47\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") " pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:34.093729 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:34.093636    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:34.370282 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:34.370249    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"]
Sep 11 17:02:34.371921 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:02:34.371894    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18e1c6ee_fd31_4311_af83_92d8d734e8bc.slice/crio-6328395c2e1aa2d620c0ed8c32526d9681be43f6b813518ef6a4a9b25061fdae WatchSource:0}: Error finding container 6328395c2e1aa2d620c0ed8c32526d9681be43f6b813518ef6a4a9b25061fdae: Status 404 returned error can't find the container with id 6328395c2e1aa2d620c0ed8c32526d9681be43f6b813518ef6a4a9b25061fdae
Sep 11 17:02:34.659892 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:34.659854    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47" event={"ID":"18e1c6ee-fd31-4311-af83-92d8d734e8bc","Type":"ContainerStarted","Data":"564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce"}
Sep 11 17:02:34.660118 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:34.659899    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47" event={"ID":"18e1c6ee-fd31-4311-af83-92d8d734e8bc","Type":"ContainerStarted","Data":"6328395c2e1aa2d620c0ed8c32526d9681be43f6b813518ef6a4a9b25061fdae"}
Sep 11 17:02:34.682596 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:34.682541    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47" podStartSLOduration=1.682522893 podStartE2EDuration="1.682522893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:02:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:02:34.68112701 +0000 UTC m=+1151.088345683" watchObservedRunningTime="2026-09-11 17:02:34.682522893 +0000 UTC m=+1151.089741541"
Sep 11 17:02:35.049258 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:35.049160    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:37.053542 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:37.053485    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:37.111849 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:37.111785    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:37.231655 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:37.231622    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:37.913987 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:37.913951    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:39.451392 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:39.451336    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:39.553657 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:39.553616    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:44.094654 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:44.094613    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:44.095111 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:44.094671    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:44.107264 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:44.107234    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:44.787708 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:44.787674    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:02:45.273701 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:45.273665    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:45.983482 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:45.983452    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:46.778244 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:46.778211    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:47.590128 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:47.590090    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:02:47.782477 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:47.782391    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:48.592112 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:48.590283    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:49.396297 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:49.396260    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:50.108762 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:50.108730    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:50.884637 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:50.884597    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:51.565499 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:51.565449    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:52.359062 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:52.359031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:52.459393 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:52.459351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:53.197379 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:53.197340    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:53.546616 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:53.546504    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:53.907951 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:53.907921    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:54.678137 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:54.678105    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:55.417094 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:55.417052    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:56.020564 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:56.020495    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f"]
Sep 11 17:02:56.024719 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:56.024680    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4268e3-29819100-88f2f"]
Sep 11 17:02:56.026228 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:56.026197    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:56.124451 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:56.124414    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:56.206859 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:56.206823    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c770ad7-9e94-485d-b71c-6cfe645e7366" path="/var/lib/kubelet/pods/5c770ad7-9e94-485d-b71c-6cfe645e7366/volumes"
Sep 11 17:02:56.241888 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:56.241427    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:56.331795 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:56.331719    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:56.634690 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:56.634653    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:56.766741 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:56.766707    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:57.465979 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:57.465939    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:58.269112 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:58.269067    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:58.869272 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:58.869239    2569 generic.go:356] "Generic (PLEG): container finished" podID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerID="74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da" exitCode=0
Sep 11 17:02:58.869729 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:58.869311    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" event={"ID":"ed4d0c50-8ffc-469f-8c86-2381dd69d887","Type":"ContainerDied","Data":"74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da"}
Sep 11 17:02:58.869809 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:58.869794    2569 scope.go:117] "RemoveContainer" containerID="74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da"
Sep 11 17:02:58.871063 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:58.871036    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:02:58.871179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:58.871117    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:02:58.871179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:58.871154    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:02:58.944853 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:58.944815    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:59.681528 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:59.681497    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:02:59.877864 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:02:59.877826    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" event={"ID":"ed4d0c50-8ffc-469f-8c86-2381dd69d887","Type":"ContainerStarted","Data":"52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2"}
Sep 11 17:03:00.231550 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:00.231501    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:03:00.233114 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:00.233060    2569 patch_prober.go:28] interesting pod/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 11 17:03:00.233278 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:00.233165    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 11 17:03:00.460663 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:00.460619    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:00.631486 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:00.631448    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:03:01.231410 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:01.231362    2569 patch_prober.go:28] interesting pod/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 11 17:03:01.231924 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:01.231440    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 11 17:03:01.335843 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:01.335799    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:01.626066 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:01.626037    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"]
Sep 11 17:03:01.717259 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:01.717230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:02.246103 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:02.231349    2569 patch_prober.go:28] interesting pod/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 11 17:03:02.246103 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:02.231411    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 11 17:03:03.231380 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:03.231338    2569 patch_prober.go:28] interesting pod/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 11 17:03:03.231765 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:03.231731    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 11 17:03:04.231570 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:04.231525    2569 patch_prober.go:28] interesting pod/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 11 17:03:04.232104 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:04.231597    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 11 17:03:05.235537 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:05.235503    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:03:05.236581 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:05.236558    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:03:05.871839 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:05.871805    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:07.048822 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:07.048786    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:07.332368 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:07.332265    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:07.453858 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:07.453825    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:09.706023 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:09.705984    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:12.364910 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:12.364876    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"]
Sep 11 17:03:12.460153 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:12.460114    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:16.665427 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.665382    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744" podUID="c663f2c4-c71c-434e-acdb-490be6e886ac" containerName="main" containerID="cri-o://511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c" gracePeriod=45
Sep 11 17:03:16.919439 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.919370    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 17:03:16.981348 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.981311    2569 generic.go:356] "Generic (PLEG): container finished" podID="c663f2c4-c71c-434e-acdb-490be6e886ac" containerID="511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c" exitCode=0
Sep 11 17:03:16.981534 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.981356    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"
Sep 11 17:03:16.981534 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.981384    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744" event={"ID":"c663f2c4-c71c-434e-acdb-490be6e886ac","Type":"ContainerDied","Data":"511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c"}
Sep 11 17:03:16.981534 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.981423    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744" event={"ID":"c663f2c4-c71c-434e-acdb-490be6e886ac","Type":"ContainerDied","Data":"122d8a628f5787d3c6a5eeaa5b99e512c11337a9f0ce800a4ca297be13350ab0"}
Sep 11 17:03:16.981534 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.981441    2569 scope.go:117] "RemoveContainer" containerID="511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c"
Sep 11 17:03:16.984303 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.984280    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-model-cache\") pod \"c663f2c4-c71c-434e-acdb-490be6e886ac\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") "
Sep 11 17:03:16.984410 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.984376    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c663f2c4-c71c-434e-acdb-490be6e886ac-tls-certs\") pod \"c663f2c4-c71c-434e-acdb-490be6e886ac\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") "
Sep 11 17:03:16.984499 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.984484    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-home\") pod \"c663f2c4-c71c-434e-acdb-490be6e886ac\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") "
Sep 11 17:03:16.984542 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.984499    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "c663f2c4-c71c-434e-acdb-490be6e886ac" (UID: "c663f2c4-c71c-434e-acdb-490be6e886ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:16.984542 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.984529    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-tmp-dir\") pod \"c663f2c4-c71c-434e-acdb-490be6e886ac\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") "
Sep 11 17:03:16.984628 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.984611    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrgxg\" (UniqueName: \"kubernetes.io/projected/c663f2c4-c71c-434e-acdb-490be6e886ac-kube-api-access-vrgxg\") pod \"c663f2c4-c71c-434e-acdb-490be6e886ac\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") "
Sep 11 17:03:16.984674 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.984637    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-dshm\") pod \"c663f2c4-c71c-434e-acdb-490be6e886ac\" (UID: \"c663f2c4-c71c-434e-acdb-490be6e886ac\") "
Sep 11 17:03:16.984861 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.984739    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-home" (OuterVolumeSpecName: "home") pod "c663f2c4-c71c-434e-acdb-490be6e886ac" (UID: "c663f2c4-c71c-434e-acdb-490be6e886ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:16.984955 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.984869    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c663f2c4-c71c-434e-acdb-490be6e886ac" (UID: "c663f2c4-c71c-434e-acdb-490be6e886ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:16.985198 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.985175    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:16.985336 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.985201    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:16.985336 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.985215    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:16.986696 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.986670    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c663f2c4-c71c-434e-acdb-490be6e886ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c663f2c4-c71c-434e-acdb-490be6e886ac" (UID: "c663f2c4-c71c-434e-acdb-490be6e886ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:03:16.986870 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.986756    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-dshm" (OuterVolumeSpecName: "dshm") pod "c663f2c4-c71c-434e-acdb-490be6e886ac" (UID: "c663f2c4-c71c-434e-acdb-490be6e886ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:16.987121 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.987101    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c663f2c4-c71c-434e-acdb-490be6e886ac-kube-api-access-vrgxg" (OuterVolumeSpecName: "kube-api-access-vrgxg") pod "c663f2c4-c71c-434e-acdb-490be6e886ac" (UID: "c663f2c4-c71c-434e-acdb-490be6e886ac"). InnerVolumeSpecName "kube-api-access-vrgxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:03:16.991767 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.991741    2569 scope.go:117] "RemoveContainer" containerID="511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c"
Sep 11 17:03:16.992168 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:03:16.992148    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c\": container with ID starting with 511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c not found: ID does not exist" containerID="511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c"
Sep 11 17:03:16.992260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:16.992178    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c"} err="failed to get container status \"511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c\": rpc error: code = NotFound desc = could not find container \"511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c\": container with ID starting with 511c5bd2caae90d48f9741f294d0d7dd7966c371daee40e6c2fa55460bf5522c not found: ID does not exist"
Sep 11 17:03:17.085850 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:17.085806    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrgxg\" (UniqueName: \"kubernetes.io/projected/c663f2c4-c71c-434e-acdb-490be6e886ac-kube-api-access-vrgxg\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:17.085850 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:17.085842    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c663f2c4-c71c-434e-acdb-490be6e886ac-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:17.085850 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:17.085852    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c663f2c4-c71c-434e-acdb-490be6e886ac-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:17.305941 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:17.305906    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"]
Sep 11 17:03:17.309470 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:17.309440    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-4268e3/test-model-4268e3-kserve-746c9d9fcc-kk744"]
Sep 11 17:03:17.782106 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:17.782045    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:03:18.206120 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:18.206069    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c663f2c4-c71c-434e-acdb-490be6e886ac" path="/var/lib/kubelet/pods/c663f2c4-c71c-434e-acdb-490be6e886ac/volumes"
Sep 11 17:03:19.065681 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:19.065652    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:24.174760 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:24.174729    2569 scope.go:117] "RemoveContainer" containerID="088e4a3b08e5aaf497bc1587c3fe4e08441669bc9b5b3cd4eb815b3577125a2d"
Sep 11 17:03:25.725488 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:25.725452    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:27.385363 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.385324    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47" podUID="18e1c6ee-fd31-4311-af83-92d8d734e8bc" containerName="main" containerID="cri-o://564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce" gracePeriod=45
Sep 11 17:03:27.650317 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.650260    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:03:27.710055 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710024    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tmp-dir\") pod \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") "
Sep 11 17:03:27.710271 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710094    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-model-cache\") pod \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") "
Sep 11 17:03:27.710271 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710153    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmjpz\" (UniqueName: \"kubernetes.io/projected/18e1c6ee-fd31-4311-af83-92d8d734e8bc-kube-api-access-fmjpz\") pod \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") "
Sep 11 17:03:27.710271 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710192    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-dshm\") pod \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") "
Sep 11 17:03:27.710271 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710225    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-home\") pod \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") "
Sep 11 17:03:27.710505 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710305    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tls-certs\") pod \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\" (UID: \"18e1c6ee-fd31-4311-af83-92d8d734e8bc\") "
Sep 11 17:03:27.710505 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710329    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "18e1c6ee-fd31-4311-af83-92d8d734e8bc" (UID: "18e1c6ee-fd31-4311-af83-92d8d734e8bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:27.710505 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710334    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-model-cache" (OuterVolumeSpecName: "model-cache") pod "18e1c6ee-fd31-4311-af83-92d8d734e8bc" (UID: "18e1c6ee-fd31-4311-af83-92d8d734e8bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:27.710670 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710506    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-home" (OuterVolumeSpecName: "home") pod "18e1c6ee-fd31-4311-af83-92d8d734e8bc" (UID: "18e1c6ee-fd31-4311-af83-92d8d734e8bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:27.710813 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710787    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:27.710813 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710815    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:27.710998 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.710830    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:27.712547 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.712522    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-dshm" (OuterVolumeSpecName: "dshm") pod "18e1c6ee-fd31-4311-af83-92d8d734e8bc" (UID: "18e1c6ee-fd31-4311-af83-92d8d734e8bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:27.712851 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.712830    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "18e1c6ee-fd31-4311-af83-92d8d734e8bc" (UID: "18e1c6ee-fd31-4311-af83-92d8d734e8bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:03:27.712951 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.712869    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e1c6ee-fd31-4311-af83-92d8d734e8bc-kube-api-access-fmjpz" (OuterVolumeSpecName: "kube-api-access-fmjpz") pod "18e1c6ee-fd31-4311-af83-92d8d734e8bc" (UID: "18e1c6ee-fd31-4311-af83-92d8d734e8bc"). InnerVolumeSpecName "kube-api-access-fmjpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:03:27.811995 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.811959    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmjpz\" (UniqueName: \"kubernetes.io/projected/18e1c6ee-fd31-4311-af83-92d8d734e8bc-kube-api-access-fmjpz\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:27.811995 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.811988    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e1c6ee-fd31-4311-af83-92d8d734e8bc-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:27.811995 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:27.811999    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e1c6ee-fd31-4311-af83-92d8d734e8bc-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:28.049811 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:28.049719    2569 generic.go:356] "Generic (PLEG): container finished" podID="18e1c6ee-fd31-4311-af83-92d8d734e8bc" containerID="564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce" exitCode=0
Sep 11 17:03:28.049811 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:28.049759    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"
Sep 11 17:03:28.050025 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:28.049810    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47" event={"ID":"18e1c6ee-fd31-4311-af83-92d8d734e8bc","Type":"ContainerDied","Data":"564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce"}
Sep 11 17:03:28.050025 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:28.049850    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47" event={"ID":"18e1c6ee-fd31-4311-af83-92d8d734e8bc","Type":"ContainerDied","Data":"6328395c2e1aa2d620c0ed8c32526d9681be43f6b813518ef6a4a9b25061fdae"}
Sep 11 17:03:28.050025 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:28.049872    2569 scope.go:117] "RemoveContainer" containerID="564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce"
Sep 11 17:03:28.061838 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:28.061813    2569 scope.go:117] "RemoveContainer" containerID="564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce"
Sep 11 17:03:28.062279 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:03:28.062254    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce\": container with ID starting with 564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce not found: ID does not exist" containerID="564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce"
Sep 11 17:03:28.062373 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:28.062288    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce"} err="failed to get container status \"564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce\": rpc error: code = NotFound desc = could not find container \"564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce\": container with ID starting with 564c40134f4f27babf432d728cb04b1bfe36e5719029b3a3e964e85df6bfe2ce not found: ID does not exist"
Sep 11 17:03:28.075897 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:28.075872    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"]
Sep 11 17:03:28.080410 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:28.080390    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-9459f0/noten-model-9459f0-kserve-6d987647-48z47"]
Sep 11 17:03:28.206260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:28.206212    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e1c6ee-fd31-4311-af83-92d8d734e8bc" path="/var/lib/kubelet/pods/18e1c6ee-fd31-4311-af83-92d8d734e8bc/volumes"
Sep 11 17:03:37.984595 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:37.984560    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:03:39.694543 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:39.694506    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:39.915717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:39.915678    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"]
Sep 11 17:03:39.915965 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:39.915938    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy" containerID="cri-o://0e965230c431a11d402b2c4e1bb33e49439a4eee417687bd9254a99d9424952f" gracePeriod=30
Sep 11 17:03:39.970631 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:39.970528    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:41.053557 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:41.053525    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:42.080816 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:42.080784    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:42.878518 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:42.878484    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:43.706017 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:43.705983    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:44.390100 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:44.390045    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:45.062143 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.062102    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:45.153453 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.153424    2569 generic.go:356] "Generic (PLEG): container finished" podID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerID="0e965230c431a11d402b2c4e1bb33e49439a4eee417687bd9254a99d9424952f" exitCode=0
Sep 11 17:03:45.153630 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.153505    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" event={"ID":"1d3de30f-d85a-4e9d-94e6-35a490a3678b","Type":"ContainerDied","Data":"0e965230c431a11d402b2c4e1bb33e49439a4eee417687bd9254a99d9424952f"}
Sep 11 17:03:45.153630 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.153584    2569 scope.go:117] "RemoveContainer" containerID="0cce37555d791bf6e2e5f44c3c471583712cf0750b30137419c3f2c76c72a4c5"
Sep 11 17:03:45.190217 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.190148    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 17:03:45.203919 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.203884    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-data\") pod \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") "
Sep 11 17:03:45.204083 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.203933    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htrhj\" (UniqueName: \"kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-kube-api-access-htrhj\") pod \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") "
Sep 11 17:03:45.204083 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.203961    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-envoy\") pod \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") "
Sep 11 17:03:45.204083 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204002    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-certs\") pod \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") "
Sep 11 17:03:45.204083 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204061    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istiod-ca-cert\") pod \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") "
Sep 11 17:03:45.204281 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204124    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-podinfo\") pod \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") "
Sep 11 17:03:45.204281 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204144    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-socket\") pod \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") "
Sep 11 17:03:45.204281 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204192    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-token\") pod \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") "
Sep 11 17:03:45.204281 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204248    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-data" (OuterVolumeSpecName: "istio-data") pod "1d3de30f-d85a-4e9d-94e6-35a490a3678b" (UID: "1d3de30f-d85a-4e9d-94e6-35a490a3678b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:45.204488 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204367    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1d3de30f-d85a-4e9d-94e6-35a490a3678b" (UID: "1d3de30f-d85a-4e9d-94e6-35a490a3678b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:45.204488 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204460    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-credential-socket\") pod \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\" (UID: \"1d3de30f-d85a-4e9d-94e6-35a490a3678b\") "
Sep 11 17:03:45.204604 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204523    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1d3de30f-d85a-4e9d-94e6-35a490a3678b" (UID: "1d3de30f-d85a-4e9d-94e6-35a490a3678b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:45.204604 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204567    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1d3de30f-d85a-4e9d-94e6-35a490a3678b" (UID: "1d3de30f-d85a-4e9d-94e6-35a490a3678b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:03:45.204837 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204788    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1d3de30f-d85a-4e9d-94e6-35a490a3678b" (UID: "1d3de30f-d85a-4e9d-94e6-35a490a3678b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:45.204993 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204970    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:45.205101 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.204999    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:45.205101 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.205012    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:45.205101 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.205041    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:45.205101 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.205086    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:45.206525 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.206502    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-kube-api-access-htrhj" (OuterVolumeSpecName: "kube-api-access-htrhj") pod "1d3de30f-d85a-4e9d-94e6-35a490a3678b" (UID: "1d3de30f-d85a-4e9d-94e6-35a490a3678b"). InnerVolumeSpecName "kube-api-access-htrhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:03:45.206597 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.206515    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-token" (OuterVolumeSpecName: "istio-token") pod "1d3de30f-d85a-4e9d-94e6-35a490a3678b" (UID: "1d3de30f-d85a-4e9d-94e6-35a490a3678b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:03:45.206706 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.206686    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1d3de30f-d85a-4e9d-94e6-35a490a3678b" (UID: "1d3de30f-d85a-4e9d-94e6-35a490a3678b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:45.206780 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.206764    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1d3de30f-d85a-4e9d-94e6-35a490a3678b" (UID: "1d3de30f-d85a-4e9d-94e6-35a490a3678b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:03:45.306137 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.306064    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:45.306137 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.306132    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:45.306137 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.306141    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htrhj\" (UniqueName: \"kubernetes.io/projected/1d3de30f-d85a-4e9d-94e6-35a490a3678b-kube-api-access-htrhj\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:45.306137 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.306152    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d3de30f-d85a-4e9d-94e6-35a490a3678b-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:03:45.964942 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:45.964822    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:46.160759 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.160731    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"
Sep 11 17:03:46.161231 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.160763    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d" event={"ID":"1d3de30f-d85a-4e9d-94e6-35a490a3678b","Type":"ContainerDied","Data":"addeb1fbe6196668d05d8a4b3deab620c93b4c7f5ec1409f5705b3accc393891"}
Sep 11 17:03:46.161231 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.160826    2569 scope.go:117] "RemoveContainer" containerID="0e965230c431a11d402b2c4e1bb33e49439a4eee417687bd9254a99d9424952f"
Sep 11 17:03:46.185919 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.185874    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"]
Sep 11 17:03:46.188351 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.188319    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4268e3-openshift-default-5c6b854cf4-9gx4d"]
Sep 11 17:03:46.206567 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.206532    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" path="/var/lib/kubelet/pods/1d3de30f-d85a-4e9d-94e6-35a490a3678b/volumes"
Sep 11 17:03:46.681364 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.681332    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"]
Sep 11 17:03:46.681776 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.681763    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy"
Sep 11 17:03:46.681776 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.681777    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy"
Sep 11 17:03:46.681902 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.681791    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e1c6ee-fd31-4311-af83-92d8d734e8bc" containerName="main"
Sep 11 17:03:46.681902 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.681796    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e1c6ee-fd31-4311-af83-92d8d734e8bc" containerName="main"
Sep 11 17:03:46.681902 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.681806    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c663f2c4-c71c-434e-acdb-490be6e886ac" containerName="main"
Sep 11 17:03:46.681902 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.681811    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c663f2c4-c71c-434e-acdb-490be6e886ac" containerName="main"
Sep 11 17:03:46.681902 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.681897    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c663f2c4-c71c-434e-acdb-490be6e886ac" containerName="main"
Sep 11 17:03:46.682093 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.681907    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy"
Sep 11 17:03:46.682093 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.681915    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy"
Sep 11 17:03:46.682093 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.681924    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e1c6ee-fd31-4311-af83-92d8d734e8bc" containerName="main"
Sep 11 17:03:46.684891 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.684864    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.687355 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.687329    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-52b0af\"/\"distinct-2-w0-52b0af-kserve-self-signed-certs\""
Sep 11 17:03:46.694126 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.694097    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"]
Sep 11 17:03:46.723021 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.722989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-tmp-dir\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.723211 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.723040    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-model-cache\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.723211 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.723180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-dshm\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.723286 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.723214    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-home\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.723286 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.723237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tqv5\" (UniqueName: \"kubernetes.io/projected/27aedd54-9f87-4259-80e1-d31fbec01c90-kube-api-access-4tqv5\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.723286 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.723271    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27aedd54-9f87-4259-80e1-d31fbec01c90-tls-certs\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.729390 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.729357    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:46.824667 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.824618    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-tmp-dir\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.824877 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.824693    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-model-cache\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.824877 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.824769    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-dshm\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.824877 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.824801    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-home\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.824877 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.824828    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tqv5\" (UniqueName: \"kubernetes.io/projected/27aedd54-9f87-4259-80e1-d31fbec01c90-kube-api-access-4tqv5\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.824877 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.824852    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27aedd54-9f87-4259-80e1-d31fbec01c90-tls-certs\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.825197 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.825070    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-tmp-dir\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.825197 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.825151    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-home\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.825197 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.825133    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-model-cache\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.827055 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.827034    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-dshm\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.827394 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.827376    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27aedd54-9f87-4259-80e1-d31fbec01c90-tls-certs\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.835506 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.835067    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tqv5\" (UniqueName: \"kubernetes.io/projected/27aedd54-9f87-4259-80e1-d31fbec01c90-kube-api-access-4tqv5\") pod \"distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") " pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:46.996197 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:46.996105    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:47.274340 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:47.274313    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"]
Sep 11 17:03:47.276255 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:03:47.276223    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27aedd54_9f87_4259_80e1_d31fbec01c90.slice/crio-83875e0c06de304c778c1654ea5e436a721e0cab16ee63b0040d16e26ebfd846 WatchSource:0}: Error finding container 83875e0c06de304c778c1654ea5e436a721e0cab16ee63b0040d16e26ebfd846: Status 404 returned error can't find the container with id 83875e0c06de304c778c1654ea5e436a721e0cab16ee63b0040d16e26ebfd846
Sep 11 17:03:48.178436 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:48.178392    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl" event={"ID":"27aedd54-9f87-4259-80e1-d31fbec01c90","Type":"ContainerStarted","Data":"645d86f391d7b0980b56ed97a28ed0186fdacf305da780997146a5a9ee911483"}
Sep 11 17:03:48.178436 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:48.178437    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl" event={"ID":"27aedd54-9f87-4259-80e1-d31fbec01c90","Type":"ContainerStarted","Data":"83875e0c06de304c778c1654ea5e436a721e0cab16ee63b0040d16e26ebfd846"}
Sep 11 17:03:48.202637 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:48.202574    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl" podStartSLOduration=2.202558004 podStartE2EDuration="2.202558004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:03:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:03:48.200248341 +0000 UTC m=+1224.607466999" watchObservedRunningTime="2026-09-11 17:03:48.202558004 +0000 UTC m=+1224.609776649"
Sep 11 17:03:50.996244 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:50.996208    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:51.159449 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:51.159417    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:51.269098 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:51.268968    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:51.377836 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:51.377795    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:51.542466 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:51.542380    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:51.695644 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:51.695613    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:52.263367 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:52.263331    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:53.136028 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:53.135993    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:54.241207 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:54.241156    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:55.005066 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:55.005021    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:55.913823 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:55.913790    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:56.996534 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:56.996492    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:56.996956 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:56.996548    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:57.009242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:57.009206    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:57.243611 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:57.243578    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:03:58.769563 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:58.769530    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:58.873174 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:58.873098    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:59.020846 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:59.020756    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"]
Sep 11 17:03:59.021538 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:59.021481    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy" containerID="cri-o://89aca449932f3dcd41515e5b375a39fe931f2dbcb8640763d8f6154de39b7b35" gracePeriod=30
Sep 11 17:03:59.056458 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:59.056426    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:59.644546 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:59.644512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:03:59.969679 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:03:59.969590    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:00.249692 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:00.249606    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:00.799233 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:00.799197    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:04:02.480084 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:02.480039    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:02.723637 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:02.723606    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:03.031199 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:03.031167    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:03.257800 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:03.257764    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:03.387496 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:03.387469    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:04.239767 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.239738    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:04.274470 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.274442    2569 generic.go:356] "Generic (PLEG): container finished" podID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerID="89aca449932f3dcd41515e5b375a39fe931f2dbcb8640763d8f6154de39b7b35" exitCode=0
Sep 11 17:04:04.274630 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.274527    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" event={"ID":"5cb65417-e286-4a09-a8ed-e2f20f0bf78f","Type":"ContainerDied","Data":"89aca449932f3dcd41515e5b375a39fe931f2dbcb8640763d8f6154de39b7b35"}
Sep 11 17:04:04.274630 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.274603    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42" event={"ID":"5cb65417-e286-4a09-a8ed-e2f20f0bf78f","Type":"ContainerDied","Data":"1914d56addc1dd6acb145ad4a5df36e81618f6d11be968644ba11a7e5434c69f"}
Sep 11 17:04:04.274735 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.274631    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1914d56addc1dd6acb145ad4a5df36e81618f6d11be968644ba11a7e5434c69f"
Sep 11 17:04:04.274735 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.274651    2569 scope.go:117] "RemoveContainer" containerID="e12b80de64597e756bc76a0c8ec01255aeb2c5fd2a74d49c0053e13ff3507e9b"
Sep 11 17:04:04.293328 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.293306    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 17:04:04.419595 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.419562    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-socket\") pod \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") "
Sep 11 17:04:04.419795 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.419631    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-data\") pod \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") "
Sep 11 17:04:04.419795 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.419650    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-envoy\") pod \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") "
Sep 11 17:04:04.419795 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.419670    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istiod-ca-cert\") pod \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") "
Sep 11 17:04:04.419795 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.419710    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-podinfo\") pod \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") "
Sep 11 17:04:04.419795 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.419748    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-credential-socket\") pod \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") "
Sep 11 17:04:04.420067 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.419803    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjr48\" (UniqueName: \"kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-kube-api-access-pjr48\") pod \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") "
Sep 11 17:04:04.420067 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.419833    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-certs\") pod \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") "
Sep 11 17:04:04.420067 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.419834    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5cb65417-e286-4a09-a8ed-e2f20f0bf78f" (UID: "5cb65417-e286-4a09-a8ed-e2f20f0bf78f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:04:04.420067 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.419950    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-token\") pod \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\" (UID: \"5cb65417-e286-4a09-a8ed-e2f20f0bf78f\") "
Sep 11 17:04:04.420067 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.419958    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-data" (OuterVolumeSpecName: "istio-data") pod "5cb65417-e286-4a09-a8ed-e2f20f0bf78f" (UID: "5cb65417-e286-4a09-a8ed-e2f20f0bf78f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:04:04.420372 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.420125    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5cb65417-e286-4a09-a8ed-e2f20f0bf78f" (UID: "5cb65417-e286-4a09-a8ed-e2f20f0bf78f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:04:04.420372 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.420143    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5cb65417-e286-4a09-a8ed-e2f20f0bf78f" (UID: "5cb65417-e286-4a09-a8ed-e2f20f0bf78f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:04:04.420510 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.420484    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:04:04.420638 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.420514    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:04:04.420638 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.420530    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:04:04.420638 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.420543    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:04:04.420827 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.420709    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5cb65417-e286-4a09-a8ed-e2f20f0bf78f" (UID: "5cb65417-e286-4a09-a8ed-e2f20f0bf78f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:04:04.422225 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.422194    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5cb65417-e286-4a09-a8ed-e2f20f0bf78f" (UID: "5cb65417-e286-4a09-a8ed-e2f20f0bf78f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:04:04.422350 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.422227    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5cb65417-e286-4a09-a8ed-e2f20f0bf78f" (UID: "5cb65417-e286-4a09-a8ed-e2f20f0bf78f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:04:04.422426 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.422405    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-token" (OuterVolumeSpecName: "istio-token") pod "5cb65417-e286-4a09-a8ed-e2f20f0bf78f" (UID: "5cb65417-e286-4a09-a8ed-e2f20f0bf78f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:04:04.422515 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.422496    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-kube-api-access-pjr48" (OuterVolumeSpecName: "kube-api-access-pjr48") pod "5cb65417-e286-4a09-a8ed-e2f20f0bf78f" (UID: "5cb65417-e286-4a09-a8ed-e2f20f0bf78f"). InnerVolumeSpecName "kube-api-access-pjr48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:04:04.521616 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.521581    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:04:04.521616 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.521612    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:04:04.521616 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.521626    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:04:04.521848 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.521636    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:04:04.521848 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.521646    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjr48\" (UniqueName: \"kubernetes.io/projected/5cb65417-e286-4a09-a8ed-e2f20f0bf78f-kube-api-access-pjr48\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:04:04.951603 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:04.951567    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:05.282125 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:05.282034    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"
Sep 11 17:04:05.318592 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:05.318561    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"]
Sep 11 17:04:05.322544 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:05.322518    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-32d680-openshift-default-79c98d5dd6-zdv42"]
Sep 11 17:04:05.718260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:05.718228    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:06.205722 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:06.205687    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" path="/var/lib/kubelet/pods/5cb65417-e286-4a09-a8ed-e2f20f0bf78f/volumes"
Sep 11 17:04:06.426423 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:06.426396    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:07.129349 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:07.129319    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:07.389382 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:07.389273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:07.493103 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:07.493041    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:07.678662 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:07.678585    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:07.783364 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:07.783325    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:24.201169 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:24.201139    2569 scope.go:117] "RemoveContainer" containerID="89aca449932f3dcd41515e5b375a39fe931f2dbcb8640763d8f6154de39b7b35"
Sep 11 17:04:24.213784 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:24.213761    2569 scope.go:117] "RemoveContainer" containerID="8ff86a6bfa604519f42f27f2531858e0022181d4f1fc9c11ed79a13bf4597fe3"
Sep 11 17:04:24.221886 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:24.221863    2569 scope.go:117] "RemoveContainer" containerID="c0decf13a2bee119bff984334b019d6406d7c2e9d1ef8dcba01327a9d47cd00a"
Sep 11 17:04:29.445260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:29.445229    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:30.206905 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:30.206873    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:30.957983 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:30.957943    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:31.914289 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:31.914254    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:32.808012 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:32.807978    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:33.118357 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.118263    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:33.382581 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.382529    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:33.673893 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.673800    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"]
Sep 11 17:04:33.674423 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.674395    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy"
Sep 11 17:04:33.674423 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.674415    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d3de30f-d85a-4e9d-94e6-35a490a3678b" containerName="istio-proxy"
Sep 11 17:04:33.674633 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.674431    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy"
Sep 11 17:04:33.674633 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.674440    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy"
Sep 11 17:04:33.674633 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.674465    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy"
Sep 11 17:04:33.674633 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.674474    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy"
Sep 11 17:04:33.674633 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.674575    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy"
Sep 11 17:04:33.679610 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.679587    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.682471 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.682451    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-327a3a-openshift-default-dockercfg-ntwwj\""
Sep 11 17:04:33.696445 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.696420    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"]
Sep 11 17:04:33.711727 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.711700    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:33.742236 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.742207    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-podinfo\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.742400 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.742248    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6008b707-b05d-4585-b319-f4cc8cf77d50-istiod-ca-cert\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.742400 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.742335    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-credential-socket\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.742479 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.742400    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-certs\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.742479 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.742435    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-token\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.742479 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.742470    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-socket\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.742585 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.742552    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-data\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.742621 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.742588    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzxrh\" (UniqueName: \"kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-kube-api-access-nzxrh\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.742621 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.742610    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-envoy\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.843869 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.843835    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6008b707-b05d-4585-b319-f4cc8cf77d50-istiod-ca-cert\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.843895    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-credential-socket\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.843952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-certs\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.843975    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-token\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.844020    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-socket\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.844069    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-data\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.844136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzxrh\" (UniqueName: \"kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-kube-api-access-nzxrh\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.844162    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-envoy\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.844243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-podinfo\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844751 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.844388    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-credential-socket\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844751 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.844450    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-socket\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844751 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.844685    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6008b707-b05d-4585-b319-f4cc8cf77d50-istiod-ca-cert\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844858 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.844771    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-certs\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.844935 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.844919    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-data\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.846540 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.846520    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-envoy\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.846690 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.846671    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-podinfo\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.852210 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.852189    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-token\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.852380 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.852362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzxrh\" (UniqueName: \"kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-kube-api-access-nzxrh\") pod \"e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") " pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:33.919719 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.919660    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:33.991349 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:33.991198    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:34.019402 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:34.019370    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:34.269039 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:34.269011    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"]
Sep 11 17:04:34.272042 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:04:34.272011    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6008b707_b05d_4585_b319_f4cc8cf77d50.slice/crio-64fb114520bf03d3062f3832389c96f2cb75f0a8bfa5c5fdcb713f1dd4d9a6d7 WatchSource:0}: Error finding container 64fb114520bf03d3062f3832389c96f2cb75f0a8bfa5c5fdcb713f1dd4d9a6d7: Status 404 returned error can't find the container with id 64fb114520bf03d3062f3832389c96f2cb75f0a8bfa5c5fdcb713f1dd4d9a6d7
Sep 11 17:04:34.275007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:34.274972    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:04:34.275136 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:34.275052    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:04:34.275136 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:34.275101    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:04:34.461172 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:34.461130    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" event={"ID":"6008b707-b05d-4585-b319-f4cc8cf77d50","Type":"ContainerStarted","Data":"f1b573cb262acf4f2bc036276050337af21a1c5f655f8b41c7a226aa7487d840"}
Sep 11 17:04:34.461385 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:34.461234    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" event={"ID":"6008b707-b05d-4585-b319-f4cc8cf77d50","Type":"ContainerStarted","Data":"64fb114520bf03d3062f3832389c96f2cb75f0a8bfa5c5fdcb713f1dd4d9a6d7"}
Sep 11 17:04:34.498211 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:34.498153    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" podStartSLOduration=1.498135048 podStartE2EDuration="1.498135048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:04:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:04:34.49270719 +0000 UTC m=+1270.899925860" watchObservedRunningTime="2026-09-11 17:04:34.498135048 +0000 UTC m=+1270.905353695"
Sep 11 17:04:34.552746 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:34.552654    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:34.991482 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:34.991444    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:35.537179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:35.537144    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:04:35.956393 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:35.956362    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:35.996733 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:35.996705    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:36.463896 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:36.463865    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:36.471919 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:36.471890    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:36.472805 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:36.472786    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:04:36.554216 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:36.554176    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:36.879566 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:36.879520    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:38.940023 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:38.939988    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:39.042890 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:39.042846    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:39.620090 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:39.620032    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:40.452287 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:40.452248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:41.242017 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:41.241983    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:42.160678 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:42.160646    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:42.244601 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:42.244567    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:42.346929 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:42.346890    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:42.678611 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:42.678576    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:42.776286 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:42.776251    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:55.133677 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.133634    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"]
Sep 11 17:04:55.134488 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.134462    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb65417-e286-4a09-a8ed-e2f20f0bf78f" containerName="istio-proxy"
Sep 11 17:04:55.137839 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.137813    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"
Sep 11 17:04:55.140002 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.139970    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:04:55.140279 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.140259    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 17:04:55.146155 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.146128    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"]
Sep 11 17:04:55.175467 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.175432    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad147d9-d656-4ef7-8b01-dbbd76ce62a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6d422395-fcxd6\" (UID: \"1ad147d9-d656-4ef7-8b01-dbbd76ce62a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"
Sep 11 17:04:55.276740 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.276700    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad147d9-d656-4ef7-8b01-dbbd76ce62a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6d422395-fcxd6\" (UID: \"1ad147d9-d656-4ef7-8b01-dbbd76ce62a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"
Sep 11 17:04:55.277419 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.277393    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad147d9-d656-4ef7-8b01-dbbd76ce62a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6d422395-fcxd6\" (UID: \"1ad147d9-d656-4ef7-8b01-dbbd76ce62a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"
Sep 11 17:04:55.455708 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.455612    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:55.458358 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.458332    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"
Sep 11 17:04:55.742339 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:55.742249    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"]
Sep 11 17:04:55.756251 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:04:55.756210    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad147d9_d656_4ef7_8b01_dbbd76ce62a6.slice/crio-cd475e7873757df38efc8c43e1c5e72478bc9f802e8ee9d94f0dc1f0b4c28208 WatchSource:0}: Error finding container cd475e7873757df38efc8c43e1c5e72478bc9f802e8ee9d94f0dc1f0b4c28208: Status 404 returned error can't find the container with id cd475e7873757df38efc8c43e1c5e72478bc9f802e8ee9d94f0dc1f0b4c28208
Sep 11 17:04:56.548788 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:56.548681    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:56.592891 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:56.592854    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6" event={"ID":"1ad147d9-d656-4ef7-8b01-dbbd76ce62a6","Type":"ContainerStarted","Data":"2f6b734a397a9facc8b8c1ee4dc5f765a5ab6f783ce03f03db0933d708f55f4b"}
Sep 11 17:04:56.593096 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:56.592906    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6" event={"ID":"1ad147d9-d656-4ef7-8b01-dbbd76ce62a6","Type":"ContainerStarted","Data":"cd475e7873757df38efc8c43e1c5e72478bc9f802e8ee9d94f0dc1f0b4c28208"}
Sep 11 17:04:56.609343 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:56.609283    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6" podStartSLOduration=1.609265717 podStartE2EDuration="1.609265717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:04:56.607750421 +0000 UTC m=+1293.014969089" watchObservedRunningTime="2026-09-11 17:04:56.609265717 +0000 UTC m=+1293.016484452"
Sep 11 17:04:57.408411 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:57.408370    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:57.601313 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:57.601274    2569 generic.go:356] "Generic (PLEG): container finished" podID="1ad147d9-d656-4ef7-8b01-dbbd76ce62a6" containerID="2f6b734a397a9facc8b8c1ee4dc5f765a5ab6f783ce03f03db0933d708f55f4b" exitCode=0
Sep 11 17:04:57.601713 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:57.601360    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6" event={"ID":"1ad147d9-d656-4ef7-8b01-dbbd76ce62a6","Type":"ContainerDied","Data":"2f6b734a397a9facc8b8c1ee4dc5f765a5ab6f783ce03f03db0933d708f55f4b"}
Sep 11 17:04:57.973388 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:57.973353    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:58.749159 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:58.749109    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"
Sep 11 17:04:58.815118 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:58.815062    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad147d9-d656-4ef7-8b01-dbbd76ce62a6-maas-api-service-ca\") pod \"1ad147d9-d656-4ef7-8b01-dbbd76ce62a6\" (UID: \"1ad147d9-d656-4ef7-8b01-dbbd76ce62a6\") "
Sep 11 17:04:58.815422 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:58.815398    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ad147d9-d656-4ef7-8b01-dbbd76ce62a6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1ad147d9-d656-4ef7-8b01-dbbd76ce62a6" (UID: "1ad147d9-d656-4ef7-8b01-dbbd76ce62a6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:04:58.815503 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:58.815490    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad147d9-d656-4ef7-8b01-dbbd76ce62a6-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:04:58.883945 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:58.883903    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:04:59.616118 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:59.616071    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"
Sep 11 17:04:59.616118 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:59.616106    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6" event={"ID":"1ad147d9-d656-4ef7-8b01-dbbd76ce62a6","Type":"ContainerDied","Data":"cd475e7873757df38efc8c43e1c5e72478bc9f802e8ee9d94f0dc1f0b4c28208"}
Sep 11 17:04:59.616325 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:59.616154    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd475e7873757df38efc8c43e1c5e72478bc9f802e8ee9d94f0dc1f0b4c28208"
Sep 11 17:04:59.643070 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:04:59.643013    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:00.340271 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:00.340236    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:01.191999 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:01.191969    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:01.456844 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:01.456762    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:01.629832 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:01.629801    2569 generic.go:356] "Generic (PLEG): container finished" podID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerID="f1b573cb262acf4f2bc036276050337af21a1c5f655f8b41c7a226aa7487d840" exitCode=0
Sep 11 17:05:01.630015 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:01.629872    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" event={"ID":"6008b707-b05d-4585-b319-f4cc8cf77d50","Type":"ContainerDied","Data":"f1b573cb262acf4f2bc036276050337af21a1c5f655f8b41c7a226aa7487d840"}
Sep 11 17:05:01.630365 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:01.630348    2569 scope.go:117] "RemoveContainer" containerID="f1b573cb262acf4f2bc036276050337af21a1c5f655f8b41c7a226aa7487d840"
Sep 11 17:05:01.631548 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:01.631520    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:05:01.631635 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:01.631589    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:05:01.631635 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:01.631620    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:05:02.119537 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:02.119505    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:02.642864 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:02.642819    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" event={"ID":"6008b707-b05d-4585-b319-f4cc8cf77d50","Type":"ContainerStarted","Data":"5fcbc3070559a96617950ba71679387b53fdabde3f4e01731e6384c8d23a2cdf"}
Sep 11 17:05:02.918909 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:02.918834    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:02.991968 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:02.991931    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:05:02.993288 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:02.993256    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 11 17:05:02.993442 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:02.993324    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 11 17:05:03.752220 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:03.752175    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:03.991616 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:03.991582    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 11 17:05:03.991803 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:03.991636    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 11 17:05:04.434047 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:04.434006    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:04.992384 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:04.992347    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 11 17:05:04.992782 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:04.992402    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 11 17:05:05.483189 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:05.483159    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:05.991823 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:05.991784    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 11 17:05:05.992057 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:05.991843    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 11 17:05:06.471973 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:06.471924    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:05:06.991867 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:06.991829    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 11 17:05:06.992063 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:06.991886    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 11 17:05:07.996650 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:07.996615    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:05:07.997830 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:07.997805    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:05:11.078335 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:11.078297    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:11.182019 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:11.181986    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:11.182896 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:11.182874    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:05:11.278313 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:11.278278    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:11.285265 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:11.285241    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"]
Sep 11 17:05:11.289577 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:11.289554    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6d422395-fcxd6"]
Sep 11 17:05:11.335151 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:11.335041    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:11.835366 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:11.835334    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:12.211977 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:12.207067    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ad147d9-d656-4ef7-8b01-dbbd76ce62a6" path="/var/lib/kubelet/pods/1ad147d9-d656-4ef7-8b01-dbbd76ce62a6/volumes"
Sep 11 17:05:12.576107 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:12.576011    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:13.286342 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:13.286294    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:14.197647 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:14.197616    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:14.594324 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:14.594240    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:14.889324 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:14.889295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:15.694856 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:15.694823    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:16.544297 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:16.544259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:19.668530 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:19.668489    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:34.693507 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.693465    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"]
Sep 11 17:05:34.694195 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.694175    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad147d9-d656-4ef7-8b01-dbbd76ce62a6" containerName="revoke-keys"
Sep 11 17:05:34.694291 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.694197    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad147d9-d656-4ef7-8b01-dbbd76ce62a6" containerName="revoke-keys"
Sep 11 17:05:34.694351 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.694321    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad147d9-d656-4ef7-8b01-dbbd76ce62a6" containerName="revoke-keys"
Sep 11 17:05:34.696501 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.696482    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.698731 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.698710    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2fae12\"/\"openshift-service-ca.crt\""
Sep 11 17:05:34.699410 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.699389    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2fae12\"/\"kube-root-ca.crt\""
Sep 11 17:05:34.699485 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.699449    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2fae12\"/\"auth-test-model-2fae12-kserve-self-signed-certs\""
Sep 11 17:05:34.699540 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.699503    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2fae12\"/\"default-dockercfg-5tgqg\""
Sep 11 17:05:34.707328 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.707304    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"]
Sep 11 17:05:34.817129 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.817069    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tls-certs\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.817317 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.817190    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-dshm\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.817317 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.817238    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-model-cache\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.817317 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.817260    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-home\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.817446 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.817324    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgrqs\" (UniqueName: \"kubernetes.io/projected/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-kube-api-access-dgrqs\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.817446 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.817405    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tmp-dir\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.868707 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.868674    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:34.918674 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.918633    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-dshm\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.918842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.918688    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-model-cache\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.918842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.918781    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-home\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.918842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.918802    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgrqs\" (UniqueName: \"kubernetes.io/projected/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-kube-api-access-dgrqs\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.919003 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.918858    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tmp-dir\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.919003 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.918925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tls-certs\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.919227 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.919207    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-model-cache\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.919296 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.919241    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-home\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.919296 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.919280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tmp-dir\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.921024 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.921005    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-dshm\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.921480 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.921458    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tls-certs\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:34.936036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:34.936011    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgrqs\" (UniqueName: \"kubernetes.io/projected/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-kube-api-access-dgrqs\") pod \"auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:35.007428 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:35.007325    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:35.285581 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:35.285555    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"]
Sep 11 17:05:35.287492 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:05:35.287466    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bfe6f3e_2eed_4f99_9d1f_3fde6b092565.slice/crio-825f147990e65b6452998ed3e1fed2c87c1b050242d10496db61552c83fba39f WatchSource:0}: Error finding container 825f147990e65b6452998ed3e1fed2c87c1b050242d10496db61552c83fba39f: Status 404 returned error can't find the container with id 825f147990e65b6452998ed3e1fed2c87c1b050242d10496db61552c83fba39f
Sep 11 17:05:35.843506 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:35.843464    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg" event={"ID":"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565","Type":"ContainerStarted","Data":"46460d323e9e57a085d45b309cbcdddfa6b9b01e864b5ff727f246d6cc5e135e"}
Sep 11 17:05:35.843899 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:35.843517    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg" event={"ID":"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565","Type":"ContainerStarted","Data":"825f147990e65b6452998ed3e1fed2c87c1b050242d10496db61552c83fba39f"}
Sep 11 17:05:35.868040 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:35.867934    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg" podStartSLOduration=1.867914801 podStartE2EDuration="1.867914801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:05:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:05:35.867711866 +0000 UTC m=+1332.274930516" watchObservedRunningTime="2026-09-11 17:05:35.867914801 +0000 UTC m=+1332.275133452"
Sep 11 17:05:35.990143 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:35.990107    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:05:37.780531 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:37.780495    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:45.007862 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:45.007820    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:45.008371 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:45.007892    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:45.020862 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:45.020827    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:45.914816 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:45.914782    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:05:49.564881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:49.564848    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:49.630158 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:49.630121    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:56.019466 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:56.019432    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:05:57.200662 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:05:57.200625    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:03.382038 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:03.382004    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:04.251244 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:04.251209    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:06.080453 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:06.080417    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:06:11.090259 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:11.090227    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"]
Sep 11 17:06:11.111322 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:11.111291    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:18.371694 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:18.371660    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:18.472256 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:18.472222    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:18.596013 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:18.595978    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"]
Sep 11 17:06:18.596290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:18.596263    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy" containerID="cri-o://52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2" gracePeriod=30
Sep 11 17:06:18.624267 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:18.624180    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:19.854956 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:19.854923    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:19.955674 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:19.955642    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:20.056571 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:20.056540    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:20.239267 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:20.239228    2569 patch_prober.go:28] interesting pod/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 17:06:20.239466 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:20.239294    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:06:20.511655 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:20.511576    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:20.733678 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:20.733636    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:20.823599 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:20.823518    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:20.923675 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:20.923641    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:22.112849 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:22.112813    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:23.844566 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.844543    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:06:23.876102 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876055    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-podinfo\") pod \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") "
Sep 11 17:06:23.876274 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876122    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-data\") pod \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") "
Sep 11 17:06:23.876274 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876173    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-credential-socket\") pod \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") "
Sep 11 17:06:23.876274 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876206    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm6pv\" (UniqueName: \"kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-kube-api-access-wm6pv\") pod \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") "
Sep 11 17:06:23.876274 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876255    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-token\") pod \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") "
Sep 11 17:06:23.876497 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876298    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-envoy\") pod \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") "
Sep 11 17:06:23.876497 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876367    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istiod-ca-cert\") pod \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") "
Sep 11 17:06:23.876497 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876405    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-socket\") pod \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") "
Sep 11 17:06:23.876497 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876432    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-certs\") pod \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\" (UID: \"ed4d0c50-8ffc-469f-8c86-2381dd69d887\") "
Sep 11 17:06:23.876497 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876443    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed4d0c50-8ffc-469f-8c86-2381dd69d887" (UID: "ed4d0c50-8ffc-469f-8c86-2381dd69d887"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:23.876719 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876490    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed4d0c50-8ffc-469f-8c86-2381dd69d887" (UID: "ed4d0c50-8ffc-469f-8c86-2381dd69d887"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:23.876719 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876614    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed4d0c50-8ffc-469f-8c86-2381dd69d887" (UID: "ed4d0c50-8ffc-469f-8c86-2381dd69d887"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:23.876811 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876743    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed4d0c50-8ffc-469f-8c86-2381dd69d887" (UID: "ed4d0c50-8ffc-469f-8c86-2381dd69d887"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:23.876868 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876848    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed4d0c50-8ffc-469f-8c86-2381dd69d887" (UID: "ed4d0c50-8ffc-469f-8c86-2381dd69d887"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:06:23.876914 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876904    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:23.876960 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876924    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:23.876960 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876938    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:23.876960 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.876950    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:23.878552 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.878527    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed4d0c50-8ffc-469f-8c86-2381dd69d887" (UID: "ed4d0c50-8ffc-469f-8c86-2381dd69d887"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:06:23.878730 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.878704    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed4d0c50-8ffc-469f-8c86-2381dd69d887" (UID: "ed4d0c50-8ffc-469f-8c86-2381dd69d887"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:23.878912 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.878894    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-kube-api-access-wm6pv" (OuterVolumeSpecName: "kube-api-access-wm6pv") pod "ed4d0c50-8ffc-469f-8c86-2381dd69d887" (UID: "ed4d0c50-8ffc-469f-8c86-2381dd69d887"). InnerVolumeSpecName "kube-api-access-wm6pv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:06:23.879054 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.879036    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed4d0c50-8ffc-469f-8c86-2381dd69d887" (UID: "ed4d0c50-8ffc-469f-8c86-2381dd69d887"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:06:23.978175 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.978140    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:23.978175 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.978171    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm6pv\" (UniqueName: \"kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-kube-api-access-wm6pv\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:23.978391 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.978186    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:23.978391 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.978198    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:23.978391 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:23.978209    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4d0c50-8ffc-469f-8c86-2381dd69d887-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:24.145935 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.145845    2569 generic.go:356] "Generic (PLEG): container finished" podID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerID="52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2" exitCode=0
Sep 11 17:06:24.145935 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.145900    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"
Sep 11 17:06:24.146147 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.145931    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" event={"ID":"ed4d0c50-8ffc-469f-8c86-2381dd69d887","Type":"ContainerDied","Data":"52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2"}
Sep 11 17:06:24.146147 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.145987    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq" event={"ID":"ed4d0c50-8ffc-469f-8c86-2381dd69d887","Type":"ContainerDied","Data":"3ca744272c5a97b7d6f376336b2a4c16f2dccf0d40b9ef7a3cc63aa534450fa4"}
Sep 11 17:06:24.146147 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.146012    2569 scope.go:117] "RemoveContainer" containerID="52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2"
Sep 11 17:06:24.156538 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.156519    2569 scope.go:117] "RemoveContainer" containerID="74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da"
Sep 11 17:06:24.164868 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.164849    2569 scope.go:117] "RemoveContainer" containerID="52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2"
Sep 11 17:06:24.165153 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:06:24.165128    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2\": container with ID starting with 52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2 not found: ID does not exist" containerID="52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2"
Sep 11 17:06:24.165203 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.165166    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2"} err="failed to get container status \"52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2\": rpc error: code = NotFound desc = could not find container \"52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2\": container with ID starting with 52c190eb20a665b08aca04aa240791c678dfa9ca53ca901a1784f9000adeafc2 not found: ID does not exist"
Sep 11 17:06:24.165203 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.165187    2569 scope.go:117] "RemoveContainer" containerID="74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da"
Sep 11 17:06:24.165555 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:06:24.165498    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da\": container with ID starting with 74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da not found: ID does not exist" containerID="74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da"
Sep 11 17:06:24.165555 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.165532    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da"} err="failed to get container status \"74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da\": rpc error: code = NotFound desc = could not find container \"74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da\": container with ID starting with 74eb9ff30369a330f14e35c677b944e8a51a507357696f525934d06eb06df0da not found: ID does not exist"
Sep 11 17:06:24.168782 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.168758    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"]
Sep 11 17:06:24.172159 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.172137    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6d422395-openshift-default-77fb9d86cb-gkxxq"]
Sep 11 17:06:24.206248 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.206221    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" path="/var/lib/kubelet/pods/ed4d0c50-8ffc-469f-8c86-2381dd69d887/volumes"
Sep 11 17:06:24.265877 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:24.265847    2569 scope.go:117] "RemoveContainer" containerID="35a9798da5a1bea430aef7c29ee784bb2d331f902c70a5561e43176eff403afc"
Sep 11 17:06:26.110222 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.110155    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7" podUID="05bd90b6-2aac-4a39-929c-28869019a323" containerName="main" containerID="cri-o://3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970" gracePeriod=45
Sep 11 17:06:26.355840 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.355813    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:06:26.398854 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.398824    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-model-cache\") pod \"05bd90b6-2aac-4a39-929c-28869019a323\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") "
Sep 11 17:06:26.399020 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.398864    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-tmp-dir\") pod \"05bd90b6-2aac-4a39-929c-28869019a323\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") "
Sep 11 17:06:26.399020 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.398894    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-home\") pod \"05bd90b6-2aac-4a39-929c-28869019a323\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") "
Sep 11 17:06:26.399020 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.398925    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-dshm\") pod \"05bd90b6-2aac-4a39-929c-28869019a323\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") "
Sep 11 17:06:26.399020 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.399015    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05bd90b6-2aac-4a39-929c-28869019a323-tls-certs\") pod \"05bd90b6-2aac-4a39-929c-28869019a323\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") "
Sep 11 17:06:26.399260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.399143    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-model-cache" (OuterVolumeSpecName: "model-cache") pod "05bd90b6-2aac-4a39-929c-28869019a323" (UID: "05bd90b6-2aac-4a39-929c-28869019a323"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:26.399260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.399160    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-home" (OuterVolumeSpecName: "home") pod "05bd90b6-2aac-4a39-929c-28869019a323" (UID: "05bd90b6-2aac-4a39-929c-28869019a323"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:26.399260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.399178    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "05bd90b6-2aac-4a39-929c-28869019a323" (UID: "05bd90b6-2aac-4a39-929c-28869019a323"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:26.399260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.399200    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjq89\" (UniqueName: \"kubernetes.io/projected/05bd90b6-2aac-4a39-929c-28869019a323-kube-api-access-rjq89\") pod \"05bd90b6-2aac-4a39-929c-28869019a323\" (UID: \"05bd90b6-2aac-4a39-929c-28869019a323\") "
Sep 11 17:06:26.399708 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.399684    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:26.399841 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.399714    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:26.399841 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.399731    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:26.401211 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.401183    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-dshm" (OuterVolumeSpecName: "dshm") pod "05bd90b6-2aac-4a39-929c-28869019a323" (UID: "05bd90b6-2aac-4a39-929c-28869019a323"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:26.401320 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.401284    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05bd90b6-2aac-4a39-929c-28869019a323-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "05bd90b6-2aac-4a39-929c-28869019a323" (UID: "05bd90b6-2aac-4a39-929c-28869019a323"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:06:26.401440 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.401418    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05bd90b6-2aac-4a39-929c-28869019a323-kube-api-access-rjq89" (OuterVolumeSpecName: "kube-api-access-rjq89") pod "05bd90b6-2aac-4a39-929c-28869019a323" (UID: "05bd90b6-2aac-4a39-929c-28869019a323"). InnerVolumeSpecName "kube-api-access-rjq89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:06:26.500413 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.500375    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05bd90b6-2aac-4a39-929c-28869019a323-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:26.500413 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.500408    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjq89\" (UniqueName: \"kubernetes.io/projected/05bd90b6-2aac-4a39-929c-28869019a323-kube-api-access-rjq89\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:26.500413 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:26.500419    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05bd90b6-2aac-4a39-929c-28869019a323-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:06:27.168047 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.168011    2569 generic.go:356] "Generic (PLEG): container finished" podID="05bd90b6-2aac-4a39-929c-28869019a323" containerID="3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970" exitCode=0
Sep 11 17:06:27.168047 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.168045    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"
Sep 11 17:06:27.168530 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.168105    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7" event={"ID":"05bd90b6-2aac-4a39-929c-28869019a323","Type":"ContainerDied","Data":"3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970"}
Sep 11 17:06:27.168530 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.168153    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7" event={"ID":"05bd90b6-2aac-4a39-929c-28869019a323","Type":"ContainerDied","Data":"4503276f7e38db4ee594bfdb54f1f2d55fcaf4dd06da759debb15eca81ce43a1"}
Sep 11 17:06:27.168530 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.168176    2569 scope.go:117] "RemoveContainer" containerID="3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970"
Sep 11 17:06:27.177826 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.177803    2569 scope.go:117] "RemoveContainer" containerID="3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970"
Sep 11 17:06:27.178165 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:06:27.178147    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970\": container with ID starting with 3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970 not found: ID does not exist" containerID="3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970"
Sep 11 17:06:27.178240 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.178175    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970"} err="failed to get container status \"3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970\": rpc error: code = NotFound desc = could not find container \"3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970\": container with ID starting with 3ab629f1416c7f3ef18c16a3b0200ff11b0ac2f62ade1af012e0e7e0a4dea970 not found: ID does not exist"
Sep 11 17:06:27.191693 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.191657    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"]
Sep 11 17:06:27.196171 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.196140    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/e2e-conflict-a-44a0e379-kserve-db4f69cdb-stzd7"]
Sep 11 17:06:27.776151 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.776121    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"]
Sep 11 17:06:27.776683 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.776660    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy"
Sep 11 17:06:27.776683 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.776682    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy"
Sep 11 17:06:27.776854 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.776702    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05bd90b6-2aac-4a39-929c-28869019a323" containerName="main"
Sep 11 17:06:27.776854 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.776708    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="05bd90b6-2aac-4a39-929c-28869019a323" containerName="main"
Sep 11 17:06:27.776854 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.776787    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="05bd90b6-2aac-4a39-929c-28869019a323" containerName="main"
Sep 11 17:06:27.776854 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.776796    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy"
Sep 11 17:06:27.776854 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.776804    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy"
Sep 11 17:06:27.779694 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.779679    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.782112 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.782089    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-52b0af\"/\"embedding-w0-52b0af-kserve-self-signed-certs\""
Sep 11 17:06:27.788722 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.788697    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"]
Sep 11 17:06:27.794576 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.794543    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:27.814378 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.814325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn68t\" (UniqueName: \"kubernetes.io/projected/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-kube-api-access-jn68t\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.814586 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.814451    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-home\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.814586 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.814486    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tmp-dir\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.814586 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.814518    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tls-certs\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.814700 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.814610    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-dshm\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.814700 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.814681    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-model-cache\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.915875 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.915814    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-dshm\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.916070 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.915913    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-model-cache\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.916070 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.915979    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn68t\" (UniqueName: \"kubernetes.io/projected/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-kube-api-access-jn68t\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.916070 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.916058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-home\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.916278 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.916213    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tmp-dir\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.916278 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.916267    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tls-certs\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.916393 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.916366    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-model-cache\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.916450 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.916437    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-home\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.916606 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.916579    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tmp-dir\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.918256 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.918229    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-dshm\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.918677 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.918656    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tls-certs\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:27.923988 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:27.923963    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn68t\" (UniqueName: \"kubernetes.io/projected/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-kube-api-access-jn68t\") pod \"embedding-w0-52b0af-kserve-76f48bdbf8-86gzn\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") " pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:28.090057 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:28.089958    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:28.206344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:28.206301    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05bd90b6-2aac-4a39-929c-28869019a323" path="/var/lib/kubelet/pods/05bd90b6-2aac-4a39-929c-28869019a323/volumes"
Sep 11 17:06:28.321366 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:28.321321    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:28.370461 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:28.370431    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"]
Sep 11 17:06:28.372006 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:06:28.371981    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21ae2a67_1d77_4a8a_ab9f_1a024201f0fa.slice/crio-d3f310e3b0d3d00d2c8f382c719bccea67cca116bfcb0ae3ed39002da713ce17 WatchSource:0}: Error finding container d3f310e3b0d3d00d2c8f382c719bccea67cca116bfcb0ae3ed39002da713ce17: Status 404 returned error can't find the container with id d3f310e3b0d3d00d2c8f382c719bccea67cca116bfcb0ae3ed39002da713ce17
Sep 11 17:06:28.731457 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:28.731422    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:28.833873 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:28.833835    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:29.183441 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:29.183406    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn" event={"ID":"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa","Type":"ContainerStarted","Data":"22d77772f7cc22d6b66d2fc61df88c0f1d52749ff4fd2d1e973e33d752c42165"}
Sep 11 17:06:29.183639 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:29.183457    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn" event={"ID":"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa","Type":"ContainerStarted","Data":"d3f310e3b0d3d00d2c8f382c719bccea67cca116bfcb0ae3ed39002da713ce17"}
Sep 11 17:06:29.397804 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:29.397772    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:33.118838 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:33.118806    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:35.408519 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:35.408476    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:35.516033 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:35.516001    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:35.613275 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:35.613242    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:38.088945 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:38.088887    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn" podStartSLOduration=11.088867591 podStartE2EDuration="11.088867591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:06:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:06:29.207105177 +0000 UTC m=+1385.614323839" watchObservedRunningTime="2026-09-11 17:06:38.088867591 +0000 UTC m=+1394.496086283"
Sep 11 17:06:38.089527 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:38.089509    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:06:38.090239 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:38.090214    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:38.090310 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:38.090263    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:38.105147 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:38.105101    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:38.248354 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:38.248322    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:06:46.124496 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:46.124440    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:46.360596 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:46.360559    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:49.116353 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:49.116314    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:06:49.698193 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:06:49.698155    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:07.281707 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:07.281669    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:13.501384 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:13.501351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:13.601647 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:13.601615    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:13.834242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:13.834152    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:14.148361 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:14.148334    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:14.339204 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:14.339175    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:16.965235 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:16.965142    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:17.081721 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:17.081686    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:17.169956 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:17.169919    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:17.999242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:17.999208    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:18.499194 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:18.499161    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:18.598900 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:18.598865    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:18.884568 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:18.884533    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:19.775144 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:19.775108    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:20.588499 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:20.588456    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:20.589241 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:20.589215    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:07:21.310423 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:21.310393    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:22.129417 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:22.129387    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:22.211264 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:22.211232    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:22.310546 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:22.310513    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:22.639775 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:22.639742    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:22.725852 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:22.725823    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:23.155187 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:23.155151    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:24.055279 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:24.055247    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:24.291367 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:24.291331    2569 scope.go:117] "RemoveContainer" containerID="7e52eb976984a18e97af4e724a689a0b93fc17c5b65fcec9c9adf007e9e3d4a1"
Sep 11 17:07:24.691513 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:24.691483    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:44.604671 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:44.604634    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:45.460930 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:45.460896    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:45.726872 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:45.726788    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:46.237861 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.237826    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:46.577199 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.577099    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"]
Sep 11 17:07:46.577554 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.577542    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy"
Sep 11 17:07:46.577597 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.577556    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed4d0c50-8ffc-469f-8c86-2381dd69d887" containerName="istio-proxy"
Sep 11 17:07:46.580331 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.580312    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.595811 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.595782    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2dc95c\"/\"distinct-2-w4-2dc95c-kserve-self-signed-certs\""
Sep 11 17:07:46.600858 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.600824    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"]
Sep 11 17:07:46.678112 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.678055    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tmp-dir\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.678112 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.678112    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tls-certs\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.678402 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.678270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djqnx\" (UniqueName: \"kubernetes.io/projected/44248117-a3d4-4b0c-a2f1-e640f1ec1472-kube-api-access-djqnx\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.678402 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.678319    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-model-cache\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.678402 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.678343    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-dshm\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.678402 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.678378    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-home\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.779965 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.779928    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tmp-dir\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.779965 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.779969    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tls-certs\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.780562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.780040    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djqnx\" (UniqueName: \"kubernetes.io/projected/44248117-a3d4-4b0c-a2f1-e640f1ec1472-kube-api-access-djqnx\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.780562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.780062    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-model-cache\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.780562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.780106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-dshm\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.780562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.780129    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-home\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.780562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.780453    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tmp-dir\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.780562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.780465    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-model-cache\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.780562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.780554    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-home\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.782629 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.782606    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-dshm\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.782738 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.782649    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tls-certs\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.788619 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.788581    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djqnx\" (UniqueName: \"kubernetes.io/projected/44248117-a3d4-4b0c-a2f1-e640f1ec1472-kube-api-access-djqnx\") pod \"distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") " pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.889301 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.889274    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:46.895392 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:46.895358    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:47.143471 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.143376    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:47.168138 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.168108    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"]
Sep 11 17:07:47.170244 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:07:47.170213    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44248117_a3d4_4b0c_a2f1_e640f1ec1472.slice/crio-62c3b495f05a8c38326f2d17c32450a392cbe5f33736353d6bb279888ca77558 WatchSource:0}: Error finding container 62c3b495f05a8c38326f2d17c32450a392cbe5f33736353d6bb279888ca77558: Status 404 returned error can't find the container with id 62c3b495f05a8c38326f2d17c32450a392cbe5f33736353d6bb279888ca77558
Sep 11 17:07:47.272928 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.272891    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:47.632579 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.632547    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:47.645273 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.645238    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk" event={"ID":"44248117-a3d4-4b0c-a2f1-e640f1ec1472","Type":"ContainerStarted","Data":"0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b"}
Sep 11 17:07:47.645438 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.645280    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk" event={"ID":"44248117-a3d4-4b0c-a2f1-e640f1ec1472","Type":"ContainerStarted","Data":"62c3b495f05a8c38326f2d17c32450a392cbe5f33736353d6bb279888ca77558"}
Sep 11 17:07:47.684623 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.684557    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk" podStartSLOduration=1.684537425 podStartE2EDuration="1.684537425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:07:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:07:47.68280684 +0000 UTC m=+1464.090025489" watchObservedRunningTime="2026-09-11 17:07:47.684537425 +0000 UTC m=+1464.091756075"
Sep 11 17:07:47.842258 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.842217    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"]
Sep 11 17:07:47.844733 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.844716    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:47.846666 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.846646    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b34361-openshift-default-dockercfg-l6wwq\""
Sep 11 17:07:47.860198 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.860164    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"]
Sep 11 17:07:47.877217 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.877182    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:47.994149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.994025    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-credential-socket\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:47.994149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.994143    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-certs\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:47.994364 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.994198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-socket\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:47.994364 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.994271    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-token\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:47.994364 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.994294    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rcvl\" (UniqueName: \"kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-kube-api-access-2rcvl\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:47.994364 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.994316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-data\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:47.994364 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.994339    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-envoy\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:47.994364 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.994364    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istiod-ca-cert\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:47.994557 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:47.994393    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-podinfo\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.095882 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.095837    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-data\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.095882 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.095888    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-envoy\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096140 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096014    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istiod-ca-cert\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096140 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096101    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-podinfo\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096224 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096176    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-credential-socket\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096273 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096250    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-certs\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096333 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096307    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-socket\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096674 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096412    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-data\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096674 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096421    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-token\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096674 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096498    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rcvl\" (UniqueName: \"kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-kube-api-access-2rcvl\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096674 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096611    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-credential-socket\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096674 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096654    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-certs\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096926 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096813    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-socket\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.096926 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.096814    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istiod-ca-cert\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.098422 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.098401    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-envoy\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.098787 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.098764    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-podinfo\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.105196 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.105174    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-token\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.105345 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.105328    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rcvl\" (UniqueName: \"kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-kube-api-access-2rcvl\") pod \"e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") " pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.155868 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.155832    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:48.268639 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.268559    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:48.371145 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.371101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:48.434436 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.434409    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"]
Sep 11 17:07:48.436589 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:07:48.436548    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2839c0a0_f2ef_405a_89ff_4c489608a3f2.slice/crio-ea7b74cafea4d790351ea329240f52668069ddd2f77ac23f1b29a572ecc8c3cd WatchSource:0}: Error finding container ea7b74cafea4d790351ea329240f52668069ddd2f77ac23f1b29a572ecc8c3cd: Status 404 returned error can't find the container with id ea7b74cafea4d790351ea329240f52668069ddd2f77ac23f1b29a572ecc8c3cd
Sep 11 17:07:48.439128 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.439067    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:07:48.439217 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.439165    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:07:48.439217 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.439195    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:07:48.655140 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.655102    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" event={"ID":"2839c0a0-f2ef-405a-89ff-4c489608a3f2","Type":"ContainerStarted","Data":"24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2"}
Sep 11 17:07:48.655362 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.655171    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" event={"ID":"2839c0a0-f2ef-405a-89ff-4c489608a3f2","Type":"ContainerStarted","Data":"ea7b74cafea4d790351ea329240f52668069ddd2f77ac23f1b29a572ecc8c3cd"}
Sep 11 17:07:48.677961 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:48.677902    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" podStartSLOduration=1.677883556 podStartE2EDuration="1.677883556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:07:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:07:48.67478522 +0000 UTC m=+1465.082003890" watchObservedRunningTime="2026-09-11 17:07:48.677883556 +0000 UTC m=+1465.085102242"
Sep 11 17:07:49.156105 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.156038    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:49.161452 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.161419    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:49.433932 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.433855    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:49.544329 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.544295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:49.639544 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.639510    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"]
Sep 11 17:07:49.642375 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.642358    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"
Sep 11 17:07:49.644883 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.644862    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 17:07:49.644981 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.644903    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:07:49.651218 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.651190    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"]
Sep 11 17:07:49.662768 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.662740    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:49.663807 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.663784    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:07:49.717056 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.716948    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6cb7f57-b95c-4886-b37b-56bb94c9d8f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh\" (UID: \"d6cb7f57-b95c-4886-b37b-56bb94c9d8f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"
Sep 11 17:07:49.818199 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.818158    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6cb7f57-b95c-4886-b37b-56bb94c9d8f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh\" (UID: \"d6cb7f57-b95c-4886-b37b-56bb94c9d8f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"
Sep 11 17:07:49.818784 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.818764    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6cb7f57-b95c-4886-b37b-56bb94c9d8f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh\" (UID: \"d6cb7f57-b95c-4886-b37b-56bb94c9d8f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"
Sep 11 17:07:49.956484 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:49.956450    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"
Sep 11 17:07:50.229383 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:50.229356    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"]
Sep 11 17:07:50.230552 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:07:50.230528    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6cb7f57_b95c_4886_b37b_56bb94c9d8f1.slice/crio-f9a1c7f01299e17711ddb1339158c606242bf4b4d8481665f9130a7e0afe417e WatchSource:0}: Error finding container f9a1c7f01299e17711ddb1339158c606242bf4b4d8481665f9130a7e0afe417e: Status 404 returned error can't find the container with id f9a1c7f01299e17711ddb1339158c606242bf4b4d8481665f9130a7e0afe417e
Sep 11 17:07:50.669752 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:50.669716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh" event={"ID":"d6cb7f57-b95c-4886-b37b-56bb94c9d8f1","Type":"ContainerStarted","Data":"c531f30df4ec9f02b617643a5b2de21ec9f2fe4efadbfb09217002a27f1f2ef9"}
Sep 11 17:07:50.669899 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:50.669781    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh" event={"ID":"d6cb7f57-b95c-4886-b37b-56bb94c9d8f1","Type":"ContainerStarted","Data":"f9a1c7f01299e17711ddb1339158c606242bf4b4d8481665f9130a7e0afe417e"}
Sep 11 17:07:51.660402 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:51.660366    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:51.677277 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:51.677245    2569 generic.go:356] "Generic (PLEG): container finished" podID="d6cb7f57-b95c-4886-b37b-56bb94c9d8f1" containerID="c531f30df4ec9f02b617643a5b2de21ec9f2fe4efadbfb09217002a27f1f2ef9" exitCode=0
Sep 11 17:07:51.677457 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:51.677334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh" event={"ID":"d6cb7f57-b95c-4886-b37b-56bb94c9d8f1","Type":"ContainerDied","Data":"c531f30df4ec9f02b617643a5b2de21ec9f2fe4efadbfb09217002a27f1f2ef9"}
Sep 11 17:07:51.762591 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:51.762559    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:52.081445 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:52.081362    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:52.182382 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:52.182351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:52.241858 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:52.241821    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:52.627171 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:52.627124    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:52.729169 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:52.729128    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:52.828457 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:52.828431    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"
Sep 11 17:07:52.851261 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:52.851223    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6cb7f57-b95c-4886-b37b-56bb94c9d8f1-maas-api-service-ca\") pod \"d6cb7f57-b95c-4886-b37b-56bb94c9d8f1\" (UID: \"d6cb7f57-b95c-4886-b37b-56bb94c9d8f1\") "
Sep 11 17:07:52.854534 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:52.854492    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6cb7f57-b95c-4886-b37b-56bb94c9d8f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d6cb7f57-b95c-4886-b37b-56bb94c9d8f1" (UID: "d6cb7f57-b95c-4886-b37b-56bb94c9d8f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:07:52.953838 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:52.953747    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6cb7f57-b95c-4886-b37b-56bb94c9d8f1-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:07:53.048422 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:53.048388    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:53.690931 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:53.690896    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"
Sep 11 17:07:53.691130 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:53.690921    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh" event={"ID":"d6cb7f57-b95c-4886-b37b-56bb94c9d8f1","Type":"ContainerDied","Data":"f9a1c7f01299e17711ddb1339158c606242bf4b4d8481665f9130a7e0afe417e"}
Sep 11 17:07:53.691130 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:53.690972    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9a1c7f01299e17711ddb1339158c606242bf4b4d8481665f9130a7e0afe417e"
Sep 11 17:07:53.819862 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:53.819834    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:54.506204 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:54.506167    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"]
Sep 11 17:07:54.521801 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:54.521767    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"]
Sep 11 17:07:54.533373 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:54.533342    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"]
Sep 11 17:07:54.545863 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:54.545830    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"]
Sep 11 17:07:54.776560 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:54.776478    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:54.878313 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:54.878277    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:55.432776 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:55.432745    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:56.458258 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:56.458228    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:56.890115 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:56.890064    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:56.890313 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:56.890137    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:56.903175 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:56.903148    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:57.223253 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:57.223171    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:57.724178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:57.724148    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:07:58.031223 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:58.031114    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:58.920782 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:58.920753    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:07:59.789637 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:59.789594    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:07:59.806412 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:07:59.806352    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:05.690266 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:05.690230    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v"]
Sep 11 17:08:05.692466 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:05.692443    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-52b0af-29819100-4qp6v"]
Sep 11 17:08:05.697121 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:05.697069    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:05.763219 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:05.763186    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:05.863004 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:05.862970    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:05.884562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:05.884526    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"]
Sep 11 17:08:05.886849 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:05.886826    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-52b0af-xxjzh"]
Sep 11 17:08:05.964816 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:05.964717    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:06.206623 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:06.206586    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e23dccc-fd2a-4533-84bf-71569464db65" path="/var/lib/kubelet/pods/2e23dccc-fd2a-4533-84bf-71569464db65/volumes"
Sep 11 17:08:06.206932 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:06.206918    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6cb7f57-b95c-4886-b37b-56bb94c9d8f1" path="/var/lib/kubelet/pods/d6cb7f57-b95c-4886-b37b-56bb94c9d8f1/volumes"
Sep 11 17:08:06.288006 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:06.287928    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:06.387894 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:06.387866    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:07.030839 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:07.030804    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:07.750315 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:07.750276    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:08.599160 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:08.599123    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:09.468325 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.468280    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:09.526370 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.526310    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl" podUID="27aedd54-9f87-4259-80e1-d31fbec01c90" containerName="main" containerID="cri-o://645d86f391d7b0980b56ed97a28ed0186fdacf305da780997146a5a9ee911483" gracePeriod=45
Sep 11 17:08:09.548635 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.548591    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn" podUID="21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" containerName="main" containerID="cri-o://22d77772f7cc22d6b66d2fc61df88c0f1d52749ff4fd2d1e973e33d752c42165" gracePeriod=45
Sep 11 17:08:09.552295 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.552265    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9" podUID="6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" containerName="main" containerID="cri-o://0d196dcd6a89b4a6fb8d5d7c03df99a9f2687c9003ee410ab05207eb91ddd2ec" gracePeriod=45
Sep 11 17:08:09.564390 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.564357    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9" podUID="87c1aebf-e5dd-4a88-a827-e79ee7beadf3" containerName="main" containerID="cri-o://b03a7305a24b08291c20944012974f7e984da4467a56b8e36de0ebd1d3d114f2" gracePeriod=45
Sep 11 17:08:09.793785 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.793752    2569 generic.go:356] "Generic (PLEG): container finished" podID="87c1aebf-e5dd-4a88-a827-e79ee7beadf3" containerID="b03a7305a24b08291c20944012974f7e984da4467a56b8e36de0ebd1d3d114f2" exitCode=0
Sep 11 17:08:09.794327 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.793866    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9" event={"ID":"87c1aebf-e5dd-4a88-a827-e79ee7beadf3","Type":"ContainerDied","Data":"b03a7305a24b08291c20944012974f7e984da4467a56b8e36de0ebd1d3d114f2"}
Sep 11 17:08:09.795999 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.795969    2569 generic.go:356] "Generic (PLEG): container finished" podID="6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" containerID="0d196dcd6a89b4a6fb8d5d7c03df99a9f2687c9003ee410ab05207eb91ddd2ec" exitCode=0
Sep 11 17:08:09.796152 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.796050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9" event={"ID":"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce","Type":"ContainerDied","Data":"0d196dcd6a89b4a6fb8d5d7c03df99a9f2687c9003ee410ab05207eb91ddd2ec"}
Sep 11 17:08:09.798531 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.798508    2569 generic.go:356] "Generic (PLEG): container finished" podID="27aedd54-9f87-4259-80e1-d31fbec01c90" containerID="645d86f391d7b0980b56ed97a28ed0186fdacf305da780997146a5a9ee911483" exitCode=0
Sep 11 17:08:09.798627 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.798592    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl" event={"ID":"27aedd54-9f87-4259-80e1-d31fbec01c90","Type":"ContainerDied","Data":"645d86f391d7b0980b56ed97a28ed0186fdacf305da780997146a5a9ee911483"}
Sep 11 17:08:09.800536 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.800513    2569 generic.go:356] "Generic (PLEG): container finished" podID="21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" containerID="22d77772f7cc22d6b66d2fc61df88c0f1d52749ff4fd2d1e973e33d752c42165" exitCode=0
Sep 11 17:08:09.800646 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.800581    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn" event={"ID":"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa","Type":"ContainerDied","Data":"22d77772f7cc22d6b66d2fc61df88c0f1d52749ff4fd2d1e973e33d752c42165"}
Sep 11 17:08:09.853322 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.853278    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:08:09.871373 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.871344    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 17:08:09.893053 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.893014    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 17:08:09.900273 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.900244    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:08:09.946652 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.946613    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-dshm\") pod \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") "
Sep 11 17:08:09.946837 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.946665    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-home\") pod \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") "
Sep 11 17:08:09.946837 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.946710    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-model-cache\") pod \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") "
Sep 11 17:08:09.946837 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.946799    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tmp-dir\") pod \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") "
Sep 11 17:08:09.947025 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.946838    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tls-certs\") pod \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") "
Sep 11 17:08:09.947025 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.946909    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tls-certs\") pod \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") "
Sep 11 17:08:09.947025 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.946950    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-model-cache\") pod \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") "
Sep 11 17:08:09.947025 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.947003    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tmp-dir\") pod \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") "
Sep 11 17:08:09.947308 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.947041    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn68t\" (UniqueName: \"kubernetes.io/projected/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-kube-api-access-jn68t\") pod \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") "
Sep 11 17:08:09.947308 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.947114    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwgz7\" (UniqueName: \"kubernetes.io/projected/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-kube-api-access-vwgz7\") pod \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") "
Sep 11 17:08:09.947308 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.947176    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-home\") pod \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\" (UID: \"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa\") "
Sep 11 17:08:09.947308 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.947235    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-dshm\") pod \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\" (UID: \"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce\") "
Sep 11 17:08:09.948559 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.946994    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" (UID: "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:09.948559 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.947109    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" (UID: "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:09.948559 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.947319    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" (UID: "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:09.948559 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.947498    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-model-cache" (OuterVolumeSpecName: "model-cache") pod "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" (UID: "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:09.948559 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.947539    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-home" (OuterVolumeSpecName: "home") pod "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" (UID: "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:09.948559 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.947733    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-home" (OuterVolumeSpecName: "home") pod "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" (UID: "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:09.950332 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.950218    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-dshm" (OuterVolumeSpecName: "dshm") pod "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" (UID: "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:09.950332 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.950224    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" (UID: "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:08:09.950332 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.950290    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" (UID: "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:08:09.950332 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.950287    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-dshm" (OuterVolumeSpecName: "dshm") pod "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" (UID: "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:09.950635 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.950597    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-kube-api-access-vwgz7" (OuterVolumeSpecName: "kube-api-access-vwgz7") pod "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" (UID: "6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce"). InnerVolumeSpecName "kube-api-access-vwgz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:08:09.950846 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:09.950825    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-kube-api-access-jn68t" (OuterVolumeSpecName: "kube-api-access-jn68t") pod "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" (UID: "21ae2a67-1d77-4a8a-ab9f-1a024201f0fa"). InnerVolumeSpecName "kube-api-access-jn68t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:08:10.048365 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048284    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-dshm\") pod \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") "
Sep 11 17:08:10.048365 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048346    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tls-certs\") pod \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") "
Sep 11 17:08:10.048365 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048362    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-model-cache\") pod \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") "
Sep 11 17:08:10.048624 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048384    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-home\") pod \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") "
Sep 11 17:08:10.048624 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048460    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tmp-dir\") pod \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") "
Sep 11 17:08:10.048624 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048474    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-model-cache\") pod \"27aedd54-9f87-4259-80e1-d31fbec01c90\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") "
Sep 11 17:08:10.048624 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048522    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-tmp-dir\") pod \"27aedd54-9f87-4259-80e1-d31fbec01c90\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") "
Sep 11 17:08:10.048624 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048546    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-dshm\") pod \"27aedd54-9f87-4259-80e1-d31fbec01c90\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") "
Sep 11 17:08:10.048624 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048606    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tqv5\" (UniqueName: \"kubernetes.io/projected/27aedd54-9f87-4259-80e1-d31fbec01c90-kube-api-access-4tqv5\") pod \"27aedd54-9f87-4259-80e1-d31fbec01c90\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") "
Sep 11 17:08:10.048947 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048648    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27aedd54-9f87-4259-80e1-d31fbec01c90-tls-certs\") pod \"27aedd54-9f87-4259-80e1-d31fbec01c90\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") "
Sep 11 17:08:10.048947 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048640    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-model-cache" (OuterVolumeSpecName: "model-cache") pod "87c1aebf-e5dd-4a88-a827-e79ee7beadf3" (UID: "87c1aebf-e5dd-4a88-a827-e79ee7beadf3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:10.048947 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048695    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq9fv\" (UniqueName: \"kubernetes.io/projected/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-kube-api-access-rq9fv\") pod \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\" (UID: \"87c1aebf-e5dd-4a88-a827-e79ee7beadf3\") "
Sep 11 17:08:10.048947 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048725    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-home\") pod \"27aedd54-9f87-4259-80e1-d31fbec01c90\" (UID: \"27aedd54-9f87-4259-80e1-d31fbec01c90\") "
Sep 11 17:08:10.048947 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048718    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-home" (OuterVolumeSpecName: "home") pod "87c1aebf-e5dd-4a88-a827-e79ee7beadf3" (UID: "87c1aebf-e5dd-4a88-a827-e79ee7beadf3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:10.048947 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048730    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "87c1aebf-e5dd-4a88-a827-e79ee7beadf3" (UID: "87c1aebf-e5dd-4a88-a827-e79ee7beadf3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:10.048947 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.048893    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "27aedd54-9f87-4259-80e1-d31fbec01c90" (UID: "27aedd54-9f87-4259-80e1-d31fbec01c90"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:10.049354 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049124    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-model-cache" (OuterVolumeSpecName: "model-cache") pod "27aedd54-9f87-4259-80e1-d31fbec01c90" (UID: "27aedd54-9f87-4259-80e1-d31fbec01c90"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:10.049354 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049140    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-home" (OuterVolumeSpecName: "home") pod "27aedd54-9f87-4259-80e1-d31fbec01c90" (UID: "27aedd54-9f87-4259-80e1-d31fbec01c90"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:10.049606 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049583    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.049680 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049612    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.049680 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049627    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.049680 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049641    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.049680 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049655    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.049680 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049667    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.049680 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049680    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.050007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049691    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.050007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049704    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn68t\" (UniqueName: \"kubernetes.io/projected/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-kube-api-access-jn68t\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.050007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049716    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwgz7\" (UniqueName: \"kubernetes.io/projected/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-kube-api-access-vwgz7\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.050007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049729    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.050007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049741    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.050007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049752    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.050007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049763    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.050007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049775    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.050007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049786    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.050007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049796    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.050007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.049808    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.051008 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.050984    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-dshm" (OuterVolumeSpecName: "dshm") pod "87c1aebf-e5dd-4a88-a827-e79ee7beadf3" (UID: "87c1aebf-e5dd-4a88-a827-e79ee7beadf3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:10.051169 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.051135    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-kube-api-access-rq9fv" (OuterVolumeSpecName: "kube-api-access-rq9fv") pod "87c1aebf-e5dd-4a88-a827-e79ee7beadf3" (UID: "87c1aebf-e5dd-4a88-a827-e79ee7beadf3"). InnerVolumeSpecName "kube-api-access-rq9fv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:08:10.051277 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.051221    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-dshm" (OuterVolumeSpecName: "dshm") pod "27aedd54-9f87-4259-80e1-d31fbec01c90" (UID: "27aedd54-9f87-4259-80e1-d31fbec01c90"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:10.051277 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.051223    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "87c1aebf-e5dd-4a88-a827-e79ee7beadf3" (UID: "87c1aebf-e5dd-4a88-a827-e79ee7beadf3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:08:10.051395 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.051320    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27aedd54-9f87-4259-80e1-d31fbec01c90-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "27aedd54-9f87-4259-80e1-d31fbec01c90" (UID: "27aedd54-9f87-4259-80e1-d31fbec01c90"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:08:10.051395 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.051377    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27aedd54-9f87-4259-80e1-d31fbec01c90-kube-api-access-4tqv5" (OuterVolumeSpecName: "kube-api-access-4tqv5") pod "27aedd54-9f87-4259-80e1-d31fbec01c90" (UID: "27aedd54-9f87-4259-80e1-d31fbec01c90"). InnerVolumeSpecName "kube-api-access-4tqv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:08:10.150471 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.150424    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq9fv\" (UniqueName: \"kubernetes.io/projected/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-kube-api-access-rq9fv\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.150471 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.150467    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.150471 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.150478    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c1aebf-e5dd-4a88-a827-e79ee7beadf3-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.150471 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.150487    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27aedd54-9f87-4259-80e1-d31fbec01c90-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.150747 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.150494    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tqv5\" (UniqueName: \"kubernetes.io/projected/27aedd54-9f87-4259-80e1-d31fbec01c90-kube-api-access-4tqv5\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.150747 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.150503    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27aedd54-9f87-4259-80e1-d31fbec01c90-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:10.200009 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.199975    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:10.807242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.807151    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"
Sep 11 17:08:10.807242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.807172    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9" event={"ID":"6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce","Type":"ContainerDied","Data":"98165028a70b4f906529e9b55e29ebe974fdb3f0e9d32e9cd35490d44e7cc454"}
Sep 11 17:08:10.807242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.807226    2569 scope.go:117] "RemoveContainer" containerID="0d196dcd6a89b4a6fb8d5d7c03df99a9f2687c9003ee410ab05207eb91ddd2ec"
Sep 11 17:08:10.808881 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.808860    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"
Sep 11 17:08:10.809007 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.808888    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl" event={"ID":"27aedd54-9f87-4259-80e1-d31fbec01c90","Type":"ContainerDied","Data":"83875e0c06de304c778c1654ea5e436a721e0cab16ee63b0040d16e26ebfd846"}
Sep 11 17:08:10.810658 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.810640    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"
Sep 11 17:08:10.810774 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.810676    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn" event={"ID":"21ae2a67-1d77-4a8a-ab9f-1a024201f0fa","Type":"ContainerDied","Data":"d3f310e3b0d3d00d2c8f382c719bccea67cca116bfcb0ae3ed39002da713ce17"}
Sep 11 17:08:10.812306 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.812289    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9" event={"ID":"87c1aebf-e5dd-4a88-a827-e79ee7beadf3","Type":"ContainerDied","Data":"3256ae18e19aa3e969e3fb5a20c74e3ba1bb37708b78bd96166df1e04160adb0"}
Sep 11 17:08:10.812395 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.812318    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"
Sep 11 17:08:10.818326 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.818305    2569 scope.go:117] "RemoveContainer" containerID="645d86f391d7b0980b56ed97a28ed0186fdacf305da780997146a5a9ee911483"
Sep 11 17:08:10.826651 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.826632    2569 scope.go:117] "RemoveContainer" containerID="22d77772f7cc22d6b66d2fc61df88c0f1d52749ff4fd2d1e973e33d752c42165"
Sep 11 17:08:10.828024 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.828006    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"]
Sep 11 17:08:10.831746 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.831727    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/premium-w0-52b0af-kserve-75c6ffdb-f7dn9"]
Sep 11 17:08:10.835320 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.835302    2569 scope.go:117] "RemoveContainer" containerID="b03a7305a24b08291c20944012974f7e984da4467a56b8e36de0ebd1d3d114f2"
Sep 11 17:08:10.840720 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.840698    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"]
Sep 11 17:08:10.846368 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.846341    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/simulator-w0-52b0af-kserve-668d7b6699-g5pn9"]
Sep 11 17:08:10.855461 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.855438    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"]
Sep 11 17:08:10.859277 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.859258    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/distinct-2-w0-52b0af-kserve-b6fbf7494-6vnbl"]
Sep 11 17:08:10.868414 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.868395    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"]
Sep 11 17:08:10.872221 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.872191    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-52b0af/embedding-w0-52b0af-kserve-76f48bdbf8-86gzn"]
Sep 11 17:08:10.980670 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:10.980640    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:11.674161 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:11.674130    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:12.206157 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:12.206058    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" path="/var/lib/kubelet/pods/21ae2a67-1d77-4a8a-ab9f-1a024201f0fa/volumes"
Sep 11 17:08:12.206607 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:12.206592    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27aedd54-9f87-4259-80e1-d31fbec01c90" path="/var/lib/kubelet/pods/27aedd54-9f87-4259-80e1-d31fbec01c90/volumes"
Sep 11 17:08:12.206974 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:12.206961    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" path="/var/lib/kubelet/pods/6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce/volumes"
Sep 11 17:08:12.207663 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:12.207647    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87c1aebf-e5dd-4a88-a827-e79ee7beadf3" path="/var/lib/kubelet/pods/87c1aebf-e5dd-4a88-a827-e79ee7beadf3/volumes"
Sep 11 17:08:12.392224 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:12.392196    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:13.015722 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:13.015687    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:13.082389 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:13.082350    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:13.236537 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:13.236509    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:13.381987 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:13.381956    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"]
Sep 11 17:08:13.382225 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:13.382197    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy" containerID="cri-o://d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f" gracePeriod=30
Sep 11 17:08:13.382424 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:13.382402    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:14.494024 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:14.493982    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:14.866427 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:14.866321    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:15.645409 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:15.645374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:15.746258 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:15.746227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:16.374141 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:16.374101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:17.227401 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:17.227374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:18.371098 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.371040    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:18.645876 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.645851    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 17:08:18.706309 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.706274    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:18.745416 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745377    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7dz9\" (UniqueName: \"kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-kube-api-access-g7dz9\") pod \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") "
Sep 11 17:08:18.745605 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745438    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-certs\") pod \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") "
Sep 11 17:08:18.745605 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745486    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-socket\") pod \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") "
Sep 11 17:08:18.745605 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745536    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-data\") pod \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") "
Sep 11 17:08:18.745605 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745562    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istiod-ca-cert\") pod \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") "
Sep 11 17:08:18.745828 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745607    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-token\") pod \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") "
Sep 11 17:08:18.745828 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745667    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-envoy\") pod \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") "
Sep 11 17:08:18.745828 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745715    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-credential-socket\") pod \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") "
Sep 11 17:08:18.745828 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745784    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-podinfo\") pod \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\" (UID: \"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91\") "
Sep 11 17:08:18.746036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745838    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" (UID: "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:18.746036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745855    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-data" (OuterVolumeSpecName: "istio-data") pod "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" (UID: "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:18.746036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745869    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" (UID: "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:18.746213 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.746092    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" (UID: "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:18.746213 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.745994    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" (UID: "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:08:18.746294 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.746246    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:18.746294 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.746266    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:18.746294 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.746280    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:18.746294 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.746291    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:18.746462 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.746302    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:18.747896 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.747874    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" (UID: "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:08:18.748036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.747866    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-token" (OuterVolumeSpecName: "istio-token") pod "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" (UID: "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:08:18.748322 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.748302    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" (UID: "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:08:18.748398 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.748308    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-kube-api-access-g7dz9" (OuterVolumeSpecName: "kube-api-access-g7dz9") pod "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" (UID: "b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91"). InnerVolumeSpecName "kube-api-access-g7dz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:08:18.832054 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832015    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"]
Sep 11 17:08:18.832483 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832468    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6cb7f57-b95c-4886-b37b-56bb94c9d8f1" containerName="revoke-keys"
Sep 11 17:08:18.832529 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832486    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6cb7f57-b95c-4886-b37b-56bb94c9d8f1" containerName="revoke-keys"
Sep 11 17:08:18.832529 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832503    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy"
Sep 11 17:08:18.832529 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832509    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy"
Sep 11 17:08:18.832529 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832519    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27aedd54-9f87-4259-80e1-d31fbec01c90" containerName="main"
Sep 11 17:08:18.832529 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832524    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="27aedd54-9f87-4259-80e1-d31fbec01c90" containerName="main"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832538    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832544    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832560    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" containerName="main"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832566    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" containerName="main"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832573    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c1aebf-e5dd-4a88-a827-e79ee7beadf3" containerName="main"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832578    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c1aebf-e5dd-4a88-a827-e79ee7beadf3" containerName="main"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832590    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" containerName="main"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832595    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" containerName="main"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832657    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="21ae2a67-1d77-4a8a-ab9f-1a024201f0fa" containerName="main"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832665    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="87c1aebf-e5dd-4a88-a827-e79ee7beadf3" containerName="main"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832674    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6cb7f57-b95c-4886-b37b-56bb94c9d8f1" containerName="revoke-keys"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832679    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e80c4fb-23a2-41f8-95ed-d5b5cbc856ce" containerName="main"
Sep 11 17:08:18.832685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832685    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="27aedd54-9f87-4259-80e1-d31fbec01c90" containerName="main"
Sep 11 17:08:18.833093 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832692    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy"
Sep 11 17:08:18.833093 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.832696    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerName="istio-proxy"
Sep 11 17:08:18.836143 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.836126    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"
Sep 11 17:08:18.838351 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.838325    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:08:18.838491 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.838390    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 17:08:18.847051 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.847007    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"]
Sep 11 17:08:18.847591 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.847553    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:18.847591 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.847580    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:18.847591 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.847593    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:18.847792 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.847606    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7dz9\" (UniqueName: \"kubernetes.io/projected/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91-kube-api-access-g7dz9\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:18.867546 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.867514    2569 generic.go:356] "Generic (PLEG): container finished" podID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" containerID="d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f" exitCode=0
Sep 11 17:08:18.867736 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.867566    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"
Sep 11 17:08:18.867736 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.867604    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" event={"ID":"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91","Type":"ContainerDied","Data":"d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f"}
Sep 11 17:08:18.867736 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.867670    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs" event={"ID":"b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91","Type":"ContainerDied","Data":"c79057d369f2c1e2247e13111979c0d3b37f4ca94d893a30d13ea74ee86b2c45"}
Sep 11 17:08:18.867736 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.867700    2569 scope.go:117] "RemoveContainer" containerID="d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f"
Sep 11 17:08:18.877390 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.877364    2569 scope.go:117] "RemoveContainer" containerID="31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a"
Sep 11 17:08:18.886307 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.886282    2569 scope.go:117] "RemoveContainer" containerID="d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f"
Sep 11 17:08:18.886769 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:08:18.886701    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f\": container with ID starting with d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f not found: ID does not exist" containerID="d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f"
Sep 11 17:08:18.886769 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.886740    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f"} err="failed to get container status \"d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f\": rpc error: code = NotFound desc = could not find container \"d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f\": container with ID starting with d087d9f924fb9a4da2667a85583ef2f15aeea590e537f106ab474ad41878c76f not found: ID does not exist"
Sep 11 17:08:18.886769 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.886765    2569 scope.go:117] "RemoveContainer" containerID="31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a"
Sep 11 17:08:18.887260 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:08:18.887145    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a\": container with ID starting with 31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a not found: ID does not exist" containerID="31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a"
Sep 11 17:08:18.887260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.887182    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a"} err="failed to get container status \"31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a\": rpc error: code = NotFound desc = could not find container \"31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a\": container with ID starting with 31a463cb914ca67c6944e6d44b7ae31496c1fcb2fe21044762d290506464fe1a not found: ID does not exist"
Sep 11 17:08:18.889388 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.889359    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"]
Sep 11 17:08:18.895062 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.895035    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-52b0af-openshift-default-6745cc867b-njnxs"]
Sep 11 17:08:18.949144 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:18.949059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4712d51-0721-4fd1-ad9c-9c5091feb9b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-935ef2-fbddq\" (UID: \"f4712d51-0721-4fd1-ad9c-9c5091feb9b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"
Sep 11 17:08:19.050420 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:19.050383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4712d51-0721-4fd1-ad9c-9c5091feb9b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-935ef2-fbddq\" (UID: \"f4712d51-0721-4fd1-ad9c-9c5091feb9b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"
Sep 11 17:08:19.051041 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:19.051023    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4712d51-0721-4fd1-ad9c-9c5091feb9b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-935ef2-fbddq\" (UID: \"f4712d51-0721-4fd1-ad9c-9c5091feb9b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"
Sep 11 17:08:19.148640 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:19.148542    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"
Sep 11 17:08:19.421762 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:19.421678    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"]
Sep 11 17:08:19.424256 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:08:19.424227    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4712d51_0721_4fd1_ad9c_9c5091feb9b2.slice/crio-8de2fe86740d35252fa597c887259b82898f65b76d56845a1aa00cda3717529d WatchSource:0}: Error finding container 8de2fe86740d35252fa597c887259b82898f65b76d56845a1aa00cda3717529d: Status 404 returned error can't find the container with id 8de2fe86740d35252fa597c887259b82898f65b76d56845a1aa00cda3717529d
Sep 11 17:08:19.533366 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:19.533332    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:19.875249 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:19.875213    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq" event={"ID":"f4712d51-0721-4fd1-ad9c-9c5091feb9b2","Type":"ContainerStarted","Data":"ce737539870809257796c03e423dbf68447cb755c6dbca3d28267953a63e6fcd"}
Sep 11 17:08:19.875458 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:19.875270    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq" event={"ID":"f4712d51-0721-4fd1-ad9c-9c5091feb9b2","Type":"ContainerStarted","Data":"8de2fe86740d35252fa597c887259b82898f65b76d56845a1aa00cda3717529d"}
Sep 11 17:08:19.891588 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:19.891533    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq" podStartSLOduration=1.8915172519999999 podStartE2EDuration="1.891517252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:08:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:08:19.890349254 +0000 UTC m=+1496.297567925" watchObservedRunningTime="2026-09-11 17:08:19.891517252 +0000 UTC m=+1496.298735899"
Sep 11 17:08:20.206228 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:20.206198    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91" path="/var/lib/kubelet/pods/b70eb72e-2b80-42b5-b6ed-ac6b9a01dc91/volumes"
Sep 11 17:08:20.382729 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:20.382698    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:20.883633 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:20.883596    2569 generic.go:356] "Generic (PLEG): container finished" podID="f4712d51-0721-4fd1-ad9c-9c5091feb9b2" containerID="ce737539870809257796c03e423dbf68447cb755c6dbca3d28267953a63e6fcd" exitCode=0
Sep 11 17:08:20.883998 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:20.883639    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq" event={"ID":"f4712d51-0721-4fd1-ad9c-9c5091feb9b2","Type":"ContainerDied","Data":"ce737539870809257796c03e423dbf68447cb755c6dbca3d28267953a63e6fcd"}
Sep 11 17:08:21.089097 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:21.089028    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:21.861978 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:21.861936    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:22.032798 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:22.032772    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"
Sep 11 17:08:22.079928 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:22.079890    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4712d51-0721-4fd1-ad9c-9c5091feb9b2-maas-api-service-ca\") pod \"f4712d51-0721-4fd1-ad9c-9c5091feb9b2\" (UID: \"f4712d51-0721-4fd1-ad9c-9c5091feb9b2\") "
Sep 11 17:08:22.080365 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:22.080331    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4712d51-0721-4fd1-ad9c-9c5091feb9b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f4712d51-0721-4fd1-ad9c-9c5091feb9b2" (UID: "f4712d51-0721-4fd1-ad9c-9c5091feb9b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:08:22.181395 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:22.181364    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4712d51-0721-4fd1-ad9c-9c5091feb9b2-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:08:22.895830 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:22.895793    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq" event={"ID":"f4712d51-0721-4fd1-ad9c-9c5091feb9b2","Type":"ContainerDied","Data":"8de2fe86740d35252fa597c887259b82898f65b76d56845a1aa00cda3717529d"}
Sep 11 17:08:22.895830 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:22.895829    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"
Sep 11 17:08:22.896122 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:22.895846    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8de2fe86740d35252fa597c887259b82898f65b76d56845a1aa00cda3717529d"
Sep 11 17:08:23.428461 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:23.428424    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:24.318119 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:24.318087    2569 scope.go:117] "RemoveContainer" containerID="953d02b982e263ba0a14f165ddf422f3e4ebe84e3b119685a608093f60a67f2c"
Sep 11 17:08:24.484193 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:24.484157    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:25.121043 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:25.121007    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:28.961308 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:28.961276    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:31.931004 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:31.930935    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:32.027848 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:32.027813    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:32.366147 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:32.366050    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:34.626705 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.626671    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:34.664568 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.664533    2569 patch_prober.go:28] interesting pod/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 11 17:08:34.664753 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.664604    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 11 17:08:34.707742 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.707670    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:34.766242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.766207    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:34.854575 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.854491    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"]
Sep 11 17:08:34.857663 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.857633    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:34.857922 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.857903    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-935ef2-fbddq"]
Sep 11 17:08:34.964158 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.964130    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:34.978193 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.978166    2569 generic.go:356] "Generic (PLEG): container finished" podID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerID="24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2" exitCode=0
Sep 11 17:08:34.978341 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.978239    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" event={"ID":"2839c0a0-f2ef-405a-89ff-4c489608a3f2","Type":"ContainerDied","Data":"24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2"}
Sep 11 17:08:34.978660 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.978644    2569 scope.go:117] "RemoveContainer" containerID="24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2"
Sep 11 17:08:34.979820 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.979795    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:08:34.979876 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.979861    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:08:34.979912 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:34.979891    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:08:35.300267 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:35.300230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:35.447330 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:35.447288    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:35.986184 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:35.986149    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" event={"ID":"2839c0a0-f2ef-405a-89ff-4c489608a3f2","Type":"ContainerStarted","Data":"a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5"}
Sep 11 17:08:36.156626 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:36.156593    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:08:36.156833 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:36.156809    2569 patch_prober.go:28] interesting pod/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 11 17:08:36.156880 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:36.156859    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 11 17:08:36.206234 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:36.206198    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4712d51-0721-4fd1-ad9c-9c5091feb9b2" path="/var/lib/kubelet/pods/f4712d51-0721-4fd1-ad9c-9c5091feb9b2/volumes"
Sep 11 17:08:36.267286 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:36.267201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:36.593096 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:36.592983    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:08:36.945658 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:36.945624    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:37.156542 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:37.156498    2569 patch_prober.go:28] interesting pod/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 11 17:08:37.157031 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:37.156574    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 11 17:08:37.787496 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:37.787461    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:38.156234 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:38.156196    2569 patch_prober.go:28] interesting pod/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 11 17:08:38.156443 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:38.156258    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 11 17:08:38.495341 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:38.495257    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:39.157095 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:39.157029    2569 patch_prober.go:28] interesting pod/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 11 17:08:39.157258 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:39.157127    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 11 17:08:39.286825 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:39.286769    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:40.030271 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:40.030236    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:40.156832 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:40.156794    2569 patch_prober.go:28] interesting pod/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 11 17:08:40.157033 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:40.156862    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 11 17:08:41.157324 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:41.157285    2569 patch_prober.go:28] interesting pod/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 11 17:08:41.157721 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:41.157349    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 11 17:08:41.341613 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:41.341573    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:41.441139 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:41.440978    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:41.616714 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:41.616681    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:42.161156 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:42.161127    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:08:42.161601 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:42.161409    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:08:42.162058 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:42.162040    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:08:43.252963 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:43.252928    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:43.993266 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:43.993230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:44.660899 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:44.660867    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:45.487501 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:45.487462    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:46.275138 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:46.275033    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:46.976403 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:46.976359    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:47.815783 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:47.815747    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:48.774169 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:48.774131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:57.788149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:57.788108    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:57.951123 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:57.951065    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:58.071443 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:58.071361    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:58.172244 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:58.172206    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:58.514531 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:58.514496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:58.605132 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:58.605058    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:08:59.285621 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:08:59.285584    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:00.091673 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:00.091636    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:00.978239 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:00.978202    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:01.668358 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:01.668303    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:02.479116 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:02.479070    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:03.706457 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:03.706421    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:09:05.111997 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:05.111964    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:05.213762 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:05.213729    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:05.354117 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:05.354068    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"]
Sep 11 17:09:05.354352 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:05.354325    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy" containerID="cri-o://a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5" gracePeriod=30
Sep 11 17:09:05.528510 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:05.528470    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:06.083112 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:06.083062    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:06.911403 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:06.911370    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:07.880129 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:07.880097    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:08.592928 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:08.592892    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:09.489185 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:09.489144    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:10.178335 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.178297    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:10.631698 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.631668    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:09:10.726597 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726505    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-certs\") pod \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") "
Sep 11 17:09:10.726764 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726625    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-socket\") pod \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") "
Sep 11 17:09:10.726764 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726663    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rcvl\" (UniqueName: \"kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-kube-api-access-2rcvl\") pod \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") "
Sep 11 17:09:10.726764 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726690    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-token\") pod \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") "
Sep 11 17:09:10.726764 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726722    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-data\") pod \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") "
Sep 11 17:09:10.726764 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726750    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-envoy\") pod \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") "
Sep 11 17:09:10.727034 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726776    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-podinfo\") pod \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") "
Sep 11 17:09:10.727034 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726801    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2839c0a0-f2ef-405a-89ff-4c489608a3f2" (UID: "2839c0a0-f2ef-405a-89ff-4c489608a3f2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:10.727034 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726808    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istiod-ca-cert\") pod \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") "
Sep 11 17:09:10.727034 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726913    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-credential-socket\") pod \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\" (UID: \"2839c0a0-f2ef-405a-89ff-4c489608a3f2\") "
Sep 11 17:09:10.727034 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726987    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-data" (OuterVolumeSpecName: "istio-data") pod "2839c0a0-f2ef-405a-89ff-4c489608a3f2" (UID: "2839c0a0-f2ef-405a-89ff-4c489608a3f2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:10.727034 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.726995    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2839c0a0-f2ef-405a-89ff-4c489608a3f2" (UID: "2839c0a0-f2ef-405a-89ff-4c489608a3f2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:10.727397 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.727252    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2839c0a0-f2ef-405a-89ff-4c489608a3f2" (UID: "2839c0a0-f2ef-405a-89ff-4c489608a3f2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:10.727397 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.727284    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2839c0a0-f2ef-405a-89ff-4c489608a3f2" (UID: "2839c0a0-f2ef-405a-89ff-4c489608a3f2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:09:10.727507 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.727446    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:10.727507 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.727467    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:10.727507 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.727480    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:10.727507 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.727493    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:10.727507 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.727505    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:10.729113 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.729065    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-token" (OuterVolumeSpecName: "istio-token") pod "2839c0a0-f2ef-405a-89ff-4c489608a3f2" (UID: "2839c0a0-f2ef-405a-89ff-4c489608a3f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:09:10.729214 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.729136    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-kube-api-access-2rcvl" (OuterVolumeSpecName: "kube-api-access-2rcvl") pod "2839c0a0-f2ef-405a-89ff-4c489608a3f2" (UID: "2839c0a0-f2ef-405a-89ff-4c489608a3f2"). InnerVolumeSpecName "kube-api-access-2rcvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:09:10.729214 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.729151    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2839c0a0-f2ef-405a-89ff-4c489608a3f2" (UID: "2839c0a0-f2ef-405a-89ff-4c489608a3f2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:10.729214 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.729157    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2839c0a0-f2ef-405a-89ff-4c489608a3f2" (UID: "2839c0a0-f2ef-405a-89ff-4c489608a3f2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:09:10.828412 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.828373    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:10.828412 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.828404    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:10.828412 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.828414    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2839c0a0-f2ef-405a-89ff-4c489608a3f2-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:10.828652 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.828423    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rcvl\" (UniqueName: \"kubernetes.io/projected/2839c0a0-f2ef-405a-89ff-4c489608a3f2-kube-api-access-2rcvl\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:10.843677 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:10.843649    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:11.194717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.194681    2569 generic.go:356] "Generic (PLEG): container finished" podID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerID="a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5" exitCode=0
Sep 11 17:09:11.195188 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.194736    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"
Sep 11 17:09:11.195188 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.194744    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" event={"ID":"2839c0a0-f2ef-405a-89ff-4c489608a3f2","Type":"ContainerDied","Data":"a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5"}
Sep 11 17:09:11.195188 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.194798    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8" event={"ID":"2839c0a0-f2ef-405a-89ff-4c489608a3f2","Type":"ContainerDied","Data":"ea7b74cafea4d790351ea329240f52668069ddd2f77ac23f1b29a572ecc8c3cd"}
Sep 11 17:09:11.195188 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.194829    2569 scope.go:117] "RemoveContainer" containerID="a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5"
Sep 11 17:09:11.204696 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.204678    2569 scope.go:117] "RemoveContainer" containerID="24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2"
Sep 11 17:09:11.213300 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.213283    2569 scope.go:117] "RemoveContainer" containerID="a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5"
Sep 11 17:09:11.213596 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:09:11.213578    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5\": container with ID starting with a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5 not found: ID does not exist" containerID="a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5"
Sep 11 17:09:11.213677 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.213605    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5"} err="failed to get container status \"a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5\": rpc error: code = NotFound desc = could not find container \"a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5\": container with ID starting with a2e9c6639aeb86cbd92e23d9688352970c445cf155d178ad879a57cf1cdb32b5 not found: ID does not exist"
Sep 11 17:09:11.213677 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.213628    2569 scope.go:117] "RemoveContainer" containerID="24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2"
Sep 11 17:09:11.213905 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:09:11.213885    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2\": container with ID starting with 24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2 not found: ID does not exist" containerID="24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2"
Sep 11 17:09:11.213946 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.213912    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2"} err="failed to get container status \"24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2\": rpc error: code = NotFound desc = could not find container \"24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2\": container with ID starting with 24661ce2aea445154ba36613cf3080c50e3da8f63f44ce6ccd5c5932a41f40a2 not found: ID does not exist"
Sep 11 17:09:11.215565 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.215543    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"]
Sep 11 17:09:11.217907 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:11.217888    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b34361-openshift-default-6b77778cd4-cwrl8"]
Sep 11 17:09:12.204940 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:12.204908    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" path="/var/lib/kubelet/pods/2839c0a0-f2ef-405a-89ff-4c489608a3f2/volumes"
Sep 11 17:09:14.829297 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:14.829258    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:16.329250 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:16.329199    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:16.431656 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:16.431616    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:16.527762 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:16.527730    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:16.592023 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:16.591938    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:16.794551 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:16.794514    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:16.895351 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:16.895322    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:16.995955 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:16.995926    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:17.695216 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:17.695171    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:18.442710 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:18.442671    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:19.199733 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:19.199701    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:19.904650 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:19.904621    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:23.698840 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:23.698789    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:23.799289 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:23.799236    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:23.947596 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:23.947565    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"]
Sep 11 17:09:23.947845 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:23.947821    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy" containerID="cri-o://7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af" gracePeriod=30
Sep 11 17:09:23.972501 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:23.972409    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:24.341953 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.341878    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"]
Sep 11 17:09:24.342318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.342304    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy"
Sep 11 17:09:24.342367 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.342319    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy"
Sep 11 17:09:24.342367 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.342328    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4712d51-0721-4fd1-ad9c-9c5091feb9b2" containerName="revoke-keys"
Sep 11 17:09:24.342367 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.342333    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4712d51-0721-4fd1-ad9c-9c5091feb9b2" containerName="revoke-keys"
Sep 11 17:09:24.342466 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.342393    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy"
Sep 11 17:09:24.342466 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.342405    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4712d51-0721-4fd1-ad9c-9c5091feb9b2" containerName="revoke-keys"
Sep 11 17:09:24.347046 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.347031    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"
Sep 11 17:09:24.349253 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.349234    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 17:09:24.349390 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.349260    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:09:24.357024 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.356994    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"]
Sep 11 17:09:24.366664 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.366639    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq\" (UID: \"0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"
Sep 11 17:09:24.467432 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.467394    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq\" (UID: \"0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"
Sep 11 17:09:24.468037 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.468018    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq\" (UID: \"0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"
Sep 11 17:09:24.483702 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.483667    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:24.658972 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.658922    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"
Sep 11 17:09:24.931934 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:24.931905    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"]
Sep 11 17:09:24.933990 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:09:24.933963    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b35d0ce_9dc6_48f5_a46d_106b3f0a61d6.slice/crio-6ba511f542addc3db5e8d69a68af1bd5d1394fa6dbac52ac99c9b982145e1594 WatchSource:0}: Error finding container 6ba511f542addc3db5e8d69a68af1bd5d1394fa6dbac52ac99c9b982145e1594: Status 404 returned error can't find the container with id 6ba511f542addc3db5e8d69a68af1bd5d1394fa6dbac52ac99c9b982145e1594
Sep 11 17:09:25.254709 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:25.254681    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:25.271138 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:25.271104    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq" event={"ID":"0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6","Type":"ContainerStarted","Data":"4f898f4af2e276ff15f1761b671bc39dadbc6fb1a6d1ce6d3a1702b02b3064d5"}
Sep 11 17:09:25.271311 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:25.271161    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq" event={"ID":"0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6","Type":"ContainerStarted","Data":"6ba511f542addc3db5e8d69a68af1bd5d1394fa6dbac52ac99c9b982145e1594"}
Sep 11 17:09:25.286191 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:25.286149    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq" podStartSLOduration=1.286136195 podStartE2EDuration="1.286136195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:09:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:09:25.285160471 +0000 UTC m=+1561.692379134" watchObservedRunningTime="2026-09-11 17:09:25.286136195 +0000 UTC m=+1561.693354842"
Sep 11 17:09:25.361086 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:25.361046    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:26.156557 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:26.156514    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:26.278766 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:26.278732    2569 generic.go:356] "Generic (PLEG): container finished" podID="0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6" containerID="4f898f4af2e276ff15f1761b671bc39dadbc6fb1a6d1ce6d3a1702b02b3064d5" exitCode=0
Sep 11 17:09:26.278919 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:26.278773    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq" event={"ID":"0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6","Type":"ContainerDied","Data":"4f898f4af2e276ff15f1761b671bc39dadbc6fb1a6d1ce6d3a1702b02b3064d5"}
Sep 11 17:09:26.788778 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:26.788744    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:27.422650 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:27.422627    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"
Sep 11 17:09:27.499521 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:27.499484    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6-maas-api-service-ca\") pod \"0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6\" (UID: \"0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6\") "
Sep 11 17:09:27.499901 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:27.499871    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6" (UID: "0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:09:27.592381 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:27.592297    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:27.600712 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:27.600686    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:28.293365 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:28.293334    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"
Sep 11 17:09:28.293531 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:28.293338    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq" event={"ID":"0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6","Type":"ContainerDied","Data":"6ba511f542addc3db5e8d69a68af1bd5d1394fa6dbac52ac99c9b982145e1594"}
Sep 11 17:09:28.293531 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:28.293456    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ba511f542addc3db5e8d69a68af1bd5d1394fa6dbac52ac99c9b982145e1594"
Sep 11 17:09:28.452475 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:28.452427    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:29.211455 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.211385    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:09:29.292286 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.292252    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:29.300556 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.300528    2569 generic.go:356] "Generic (PLEG): container finished" podID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerID="7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af" exitCode=0
Sep 11 17:09:29.300702 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.300596    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"
Sep 11 17:09:29.300702 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.300604    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" event={"ID":"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb","Type":"ContainerDied","Data":"7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af"}
Sep 11 17:09:29.300702 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.300661    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6" event={"ID":"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb","Type":"ContainerDied","Data":"95d99c963652f8f6c9279489f04726fc5badbf12ad81343b84ff64e55fafff83"}
Sep 11 17:09:29.300702 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.300689    2569 scope.go:117] "RemoveContainer" containerID="7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af"
Sep 11 17:09:29.310399 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.310377    2569 scope.go:117] "RemoveContainer" containerID="d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1"
Sep 11 17:09:29.316409 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.316381    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-certs\") pod \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") "
Sep 11 17:09:29.316538 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.316413    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-token\") pod \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") "
Sep 11 17:09:29.316538 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.316452    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brtbt\" (UniqueName: \"kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-kube-api-access-brtbt\") pod \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") "
Sep 11 17:09:29.316538 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.316481    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-podinfo\") pod \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") "
Sep 11 17:09:29.316725 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.316556    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-socket\") pod \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") "
Sep 11 17:09:29.316725 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.316591    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-credential-socket\") pod \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") "
Sep 11 17:09:29.316725 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.316647    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" (UID: "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:29.316908 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.316881    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" (UID: "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:29.316997 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.316974    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" (UID: "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:29.317063 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.317007    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-envoy\") pod \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") "
Sep 11 17:09:29.317139 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.317092    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istiod-ca-cert\") pod \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") "
Sep 11 17:09:29.317139 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.317132    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-data\") pod \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\" (UID: \"70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb\") "
Sep 11 17:09:29.317590 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.317569    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:29.318016 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.318000    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:29.318253 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.318228    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:29.318253 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.317938    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" (UID: "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:29.318434 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.318045    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" (UID: "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:09:29.319210 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.319183    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-kube-api-access-brtbt" (OuterVolumeSpecName: "kube-api-access-brtbt") pod "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" (UID: "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb"). InnerVolumeSpecName "kube-api-access-brtbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:09:29.319305 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.319246    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" (UID: "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:09:29.319458 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.319439    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" (UID: "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:09:29.319641 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.319620    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" (UID: "70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:29.321446 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.321426    2569 scope.go:117] "RemoveContainer" containerID="7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af"
Sep 11 17:09:29.321804 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:09:29.321781    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af\": container with ID starting with 7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af not found: ID does not exist" containerID="7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af"
Sep 11 17:09:29.321878 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.321819    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af"} err="failed to get container status \"7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af\": rpc error: code = NotFound desc = could not find container \"7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af\": container with ID starting with 7b82b1eb43b5e456fa5806572aca72649926c6ad094fc2fd062797f40ebc53af not found: ID does not exist"
Sep 11 17:09:29.321878 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.321844    2569 scope.go:117] "RemoveContainer" containerID="d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1"
Sep 11 17:09:29.322196 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:09:29.322175    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1\": container with ID starting with d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1 not found: ID does not exist" containerID="d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1"
Sep 11 17:09:29.322261 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.322204    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1"} err="failed to get container status \"d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1\": rpc error: code = NotFound desc = could not find container \"d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1\": container with ID starting with d349ec74f4b09006c8b0a266b4f5dcfd7333060eb43ce235080012ce608c28e1 not found: ID does not exist"
Sep 11 17:09:29.419470 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.419419    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:29.419689 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.419481    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:29.419689 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.419496    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:29.419689 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.419508    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:29.419689 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.419522    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brtbt\" (UniqueName: \"kubernetes.io/projected/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-kube-api-access-brtbt\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:29.419689 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.419540    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:29.624758 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.624718    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"]
Sep 11 17:09:29.631191 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:29.631149    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b6653-openshift-default-7b589686c8-4s5v6"]
Sep 11 17:09:30.206443 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:30.206410    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" path="/var/lib/kubelet/pods/70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb/volumes"
Sep 11 17:09:30.976807 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:30.976770    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:09:34.582029 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:34.581991    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:35.339423 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:35.339393    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:35.439629 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:35.439573    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:35.512852 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:35.512819    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"]
Sep 11 17:09:35.515697 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:35.515670    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff65f-n9dtq"]
Sep 11 17:09:35.518709 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:35.518675    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:35.673613 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:35.673582    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:35.893488 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:35.893445    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:35.992152 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:35.992015    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:36.206025 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:36.205980    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6" path="/var/lib/kubelet/pods/0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6/volumes"
Sep 11 17:09:36.688069 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:36.688027    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:37.387547 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:37.387513    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:38.138663 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:38.138618    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:42.067690 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:42.067656    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:42.167368 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:42.167331    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:42.373433 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:42.373351    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"]
Sep 11 17:09:42.373595 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:42.373559    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy" containerID="cri-o://2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad" gracePeriod=30
Sep 11 17:09:42.391449 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:42.391419    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:09:43.229648 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:43.229585    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 17:09:43.229648 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:43.229658    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:09:47.636852 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.636786    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:09:47.813281 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813247    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-certs\") pod \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") "
Sep 11 17:09:47.813281 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813282    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-envoy\") pod \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") "
Sep 11 17:09:47.813509 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813342    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-socket\") pod \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") "
Sep 11 17:09:47.813509 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813378    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istiod-ca-cert\") pod \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") "
Sep 11 17:09:47.813509 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813429    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plz48\" (UniqueName: \"kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-kube-api-access-plz48\") pod \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") "
Sep 11 17:09:47.813509 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813474    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-token\") pod \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") "
Sep 11 17:09:47.813509 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813489    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-podinfo\") pod \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") "
Sep 11 17:09:47.813773 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813524    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-credential-socket\") pod \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") "
Sep 11 17:09:47.813773 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813557    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-data\") pod \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\" (UID: \"f9e34fc3-2c43-4c18-9445-c344c60ecc86\") "
Sep 11 17:09:47.813773 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813592    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f9e34fc3-2c43-4c18-9445-c344c60ecc86" (UID: "f9e34fc3-2c43-4c18-9445-c344c60ecc86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:47.813773 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813660    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f9e34fc3-2c43-4c18-9445-c344c60ecc86" (UID: "f9e34fc3-2c43-4c18-9445-c344c60ecc86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:47.813998 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813874    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-data" (OuterVolumeSpecName: "istio-data") pod "f9e34fc3-2c43-4c18-9445-c344c60ecc86" (UID: "f9e34fc3-2c43-4c18-9445-c344c60ecc86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:47.813998 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813973    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f9e34fc3-2c43-4c18-9445-c344c60ecc86" (UID: "f9e34fc3-2c43-4c18-9445-c344c60ecc86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:47.814149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.813978    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:47.814149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.814035    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:47.814149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.814050    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:47.814149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.814106    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f9e34fc3-2c43-4c18-9445-c344c60ecc86" (UID: "f9e34fc3-2c43-4c18-9445-c344c60ecc86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:09:47.815655 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.815625    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f9e34fc3-2c43-4c18-9445-c344c60ecc86" (UID: "f9e34fc3-2c43-4c18-9445-c344c60ecc86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:09:47.815770 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.815726    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9e34fc3-2c43-4c18-9445-c344c60ecc86" (UID: "f9e34fc3-2c43-4c18-9445-c344c60ecc86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:09:47.816122 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.816098    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f9e34fc3-2c43-4c18-9445-c344c60ecc86" (UID: "f9e34fc3-2c43-4c18-9445-c344c60ecc86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:09:47.816273 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.816095    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-kube-api-access-plz48" (OuterVolumeSpecName: "kube-api-access-plz48") pod "f9e34fc3-2c43-4c18-9445-c344c60ecc86" (UID: "f9e34fc3-2c43-4c18-9445-c344c60ecc86"). InnerVolumeSpecName "kube-api-access-plz48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:09:47.915504 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.915415    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:47.915504 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.915446    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:47.915504 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.915454    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:47.915504 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.915463    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:47.915504 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.915473    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9e34fc3-2c43-4c18-9445-c344c60ecc86-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:47.915504 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:47.915481    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plz48\" (UniqueName: \"kubernetes.io/projected/f9e34fc3-2c43-4c18-9445-c344c60ecc86-kube-api-access-plz48\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:09:48.412941 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.412902    2569 generic.go:356] "Generic (PLEG): container finished" podID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerID="2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad" exitCode=0
Sep 11 17:09:48.413112 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.412965    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"
Sep 11 17:09:48.413112 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.412988    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" event={"ID":"f9e34fc3-2c43-4c18-9445-c344c60ecc86","Type":"ContainerDied","Data":"2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad"}
Sep 11 17:09:48.413112 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.413109    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7" event={"ID":"f9e34fc3-2c43-4c18-9445-c344c60ecc86","Type":"ContainerDied","Data":"e9278af24f57f297686856fad4e8c9d59fde60ff16630fe9ee705ffe3baa4d05"}
Sep 11 17:09:48.413240 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.413140    2569 scope.go:117] "RemoveContainer" containerID="2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad"
Sep 11 17:09:48.422267 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.422248    2569 scope.go:117] "RemoveContainer" containerID="d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b"
Sep 11 17:09:48.433746 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.433721    2569 scope.go:117] "RemoveContainer" containerID="2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad"
Sep 11 17:09:48.433876 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.433862    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"]
Sep 11 17:09:48.434287 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:09:48.434255    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad\": container with ID starting with 2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad not found: ID does not exist" containerID="2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad"
Sep 11 17:09:48.434450 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.434420    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad"} err="failed to get container status \"2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad\": rpc error: code = NotFound desc = could not find container \"2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad\": container with ID starting with 2e720e7cb013263c11e76eae5c73811aec015a8e9dcc19a1378d03274cd2b3ad not found: ID does not exist"
Sep 11 17:09:48.434450 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.434451    2569 scope.go:117] "RemoveContainer" containerID="d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b"
Sep 11 17:09:48.434773 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:09:48.434753    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b\": container with ID starting with d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b not found: ID does not exist" containerID="d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b"
Sep 11 17:09:48.434857 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.434781    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b"} err="failed to get container status \"d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b\": rpc error: code = NotFound desc = could not find container \"d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b\": container with ID starting with d7c53a4355b013a8ffd6e55da1a274d48dbee0723facdf8350bc18ca2acb995b not found: ID does not exist"
Sep 11 17:09:48.436187 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:48.436168    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff65f-openshift-default-cfdb9fc5d-hg4k7"]
Sep 11 17:09:50.206141 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:50.206103    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" path="/var/lib/kubelet/pods/f9e34fc3-2c43-4c18-9445-c344c60ecc86/volumes"
Sep 11 17:09:53.787170 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:09:53.787132    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:10:03.878479 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:03.878445    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:13.882113 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:13.882064    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:10:20.465505 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:20.465415    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"]
Sep 11 17:10:20.488951 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:20.488922    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:21.789223 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:21.789196    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:28.752474 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.752437    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:28.860598 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.860563    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"]
Sep 11 17:10:28.860987 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.860975    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy"
Sep 11 17:10:28.861036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.860989    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy"
Sep 11 17:10:28.861036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861000    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy"
Sep 11 17:10:28.861036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861006    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy"
Sep 11 17:10:28.861036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861019    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy"
Sep 11 17:10:28.861036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861024    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy"
Sep 11 17:10:28.861036 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861035    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy"
Sep 11 17:10:28.861285 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861040    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy"
Sep 11 17:10:28.861285 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861047    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6" containerName="revoke-keys"
Sep 11 17:10:28.861285 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861052    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6" containerName="revoke-keys"
Sep 11 17:10:28.861285 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861067    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy"
Sep 11 17:10:28.861285 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861086    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy"
Sep 11 17:10:28.861285 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861154    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy"
Sep 11 17:10:28.861285 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861161    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2839c0a0-f2ef-405a-89ff-4c489608a3f2" containerName="istio-proxy"
Sep 11 17:10:28.861285 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861168    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy"
Sep 11 17:10:28.861285 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.861174    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b35d0ce-9dc6-48f5-a46d-106b3f0a61d6" containerName="revoke-keys"
Sep 11 17:10:28.864270 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.864254    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"
Sep 11 17:10:28.866235 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.866214    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:10:28.866366 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.866214    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 17:10:28.871677 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.871458    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"]
Sep 11 17:10:28.928379 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:28.928340    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6431688b-a847-47da-9aa4-5531eccf0268-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b\" (UID: \"6431688b-a847-47da-9aa4-5531eccf0268\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"
Sep 11 17:10:29.029650 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:29.029559    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6431688b-a847-47da-9aa4-5531eccf0268-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b\" (UID: \"6431688b-a847-47da-9aa4-5531eccf0268\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"
Sep 11 17:10:29.030270 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:29.030247    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6431688b-a847-47da-9aa4-5531eccf0268-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b\" (UID: \"6431688b-a847-47da-9aa4-5531eccf0268\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"
Sep 11 17:10:29.163642 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:29.163609    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:29.175524 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:29.175491    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"
Sep 11 17:10:29.455119 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:29.455090    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"]
Sep 11 17:10:29.457777 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:10:29.457745    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6431688b_a847_47da_9aa4_5531eccf0268.slice/crio-4c918ef8abecddb98d2fcb8ccca6d2e21c7c242432ccf730fec949dab5cdaccc WatchSource:0}: Error finding container 4c918ef8abecddb98d2fcb8ccca6d2e21c7c242432ccf730fec949dab5cdaccc: Status 404 returned error can't find the container with id 4c918ef8abecddb98d2fcb8ccca6d2e21c7c242432ccf730fec949dab5cdaccc
Sep 11 17:10:29.645587 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:29.645551    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b" event={"ID":"6431688b-a847-47da-9aa4-5531eccf0268","Type":"ContainerStarted","Data":"4c918ef8abecddb98d2fcb8ccca6d2e21c7c242432ccf730fec949dab5cdaccc"}
Sep 11 17:10:30.079817 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:30.079723    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:30.652399 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:30.652362    2569 generic.go:356] "Generic (PLEG): container finished" podID="6431688b-a847-47da-9aa4-5531eccf0268" containerID="8b25170d5b74116a8ad1a31e9dfea98309d610aa3f39ed21ff8c3b83a9ef92a1" exitCode=0
Sep 11 17:10:30.652601 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:30.652450    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b" event={"ID":"6431688b-a847-47da-9aa4-5531eccf0268","Type":"ContainerDied","Data":"8b25170d5b74116a8ad1a31e9dfea98309d610aa3f39ed21ff8c3b83a9ef92a1"}
Sep 11 17:10:30.789688 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:30.789655    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:30.971927 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:30.971830    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"]
Sep 11 17:10:30.972435 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:30.972415    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="70deb7d5-a0b5-4ce1-ac24-e0c19619c8cb" containerName="istio-proxy"
Sep 11 17:10:30.972543 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:30.972436    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9e34fc3-2c43-4c18-9445-c344c60ecc86" containerName="istio-proxy"
Sep 11 17:10:30.975747 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:30.975722    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:30.977717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:30.977683    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2fae12\"/\"sub-test-model-2fae12-kserve-self-signed-certs\""
Sep 11 17:10:30.985099 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:30.985037    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"]
Sep 11 17:10:31.002138 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.002069    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:31.049922 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.049875    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-home\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.050178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.049947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-model-cache\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.050178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.050031    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-tmp-dir\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.050178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.050071    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-dshm\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.050178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.050168    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48688c98-f49e-48c2-a01c-924641d320a2-tls-certs\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.050390 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.050242    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm4lh\" (UniqueName: \"kubernetes.io/projected/48688c98-f49e-48c2-a01c-924641d320a2-kube-api-access-wm4lh\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.151527 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.151488    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48688c98-f49e-48c2-a01c-924641d320a2-tls-certs\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.152006 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.151554    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm4lh\" (UniqueName: \"kubernetes.io/projected/48688c98-f49e-48c2-a01c-924641d320a2-kube-api-access-wm4lh\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.152006 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.151606    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-home\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.152006 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.151665    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-model-cache\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.152006 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.151709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-tmp-dir\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.152006 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.151758    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-dshm\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.152327 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.152206    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-model-cache\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.152327 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.152289    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-tmp-dir\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.152439 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.152327    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-home\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.154281 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.154258    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-dshm\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.154570 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.154549    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48688c98-f49e-48c2-a01c-924641d320a2-tls-certs\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.159820 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.159791    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm4lh\" (UniqueName: \"kubernetes.io/projected/48688c98-f49e-48c2-a01c-924641d320a2-kube-api-access-wm4lh\") pod \"sub-test-model-2fae12-kserve-65cdb486f-9xx7x\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.288360 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.288266    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:31.561475 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.561448    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"]
Sep 11 17:10:31.563122 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:10:31.563090    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48688c98_f49e_48c2_a01c_924641d320a2.slice/crio-80021c8c2c6cfdd32a18b4de66cdccb6f24a3932f0edc2a4e55f25ffcd0cf5b1 WatchSource:0}: Error finding container 80021c8c2c6cfdd32a18b4de66cdccb6f24a3932f0edc2a4e55f25ffcd0cf5b1: Status 404 returned error can't find the container with id 80021c8c2c6cfdd32a18b4de66cdccb6f24a3932f0edc2a4e55f25ffcd0cf5b1
Sep 11 17:10:31.653262 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.653218    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:31.659828 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.659788    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x" event={"ID":"48688c98-f49e-48c2-a01c-924641d320a2","Type":"ContainerStarted","Data":"576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa"}
Sep 11 17:10:31.659828 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.659839    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x" event={"ID":"48688c98-f49e-48c2-a01c-924641d320a2","Type":"ContainerStarted","Data":"80021c8c2c6cfdd32a18b4de66cdccb6f24a3932f0edc2a4e55f25ffcd0cf5b1"}
Sep 11 17:10:31.677192 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.677131    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x" podStartSLOduration=1.677116463 podStartE2EDuration="1.677116463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:10:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:10:31.675562038 +0000 UTC m=+1628.082780721" watchObservedRunningTime="2026-09-11 17:10:31.677116463 +0000 UTC m=+1628.084335111"
Sep 11 17:10:31.782994 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.782967    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"
Sep 11 17:10:31.859403 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.859308    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6431688b-a847-47da-9aa4-5531eccf0268-maas-api-service-ca\") pod \"6431688b-a847-47da-9aa4-5531eccf0268\" (UID: \"6431688b-a847-47da-9aa4-5531eccf0268\") "
Sep 11 17:10:31.859656 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.859631    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6431688b-a847-47da-9aa4-5531eccf0268-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6431688b-a847-47da-9aa4-5531eccf0268" (UID: "6431688b-a847-47da-9aa4-5531eccf0268"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:10:31.961033 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:31.960988    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6431688b-a847-47da-9aa4-5531eccf0268-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:32.380838 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:32.380802    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:32.667291 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:32.667245    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"
Sep 11 17:10:32.667291 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:32.667280    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b" event={"ID":"6431688b-a847-47da-9aa4-5531eccf0268","Type":"ContainerDied","Data":"4c918ef8abecddb98d2fcb8ccca6d2e21c7c242432ccf730fec949dab5cdaccc"}
Sep 11 17:10:32.667546 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:32.667350    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c918ef8abecddb98d2fcb8ccca6d2e21c7c242432ccf730fec949dab5cdaccc"
Sep 11 17:10:33.732238 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:33.732200    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"]
Sep 11 17:10:33.744205 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:33.744177    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"]
Sep 11 17:10:33.751858 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:33.751830    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"]
Sep 11 17:10:33.802656 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:33.802624    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:35.484933 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.484887    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg" podUID="6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" containerName="main" containerID="cri-o://46460d323e9e57a085d45b309cbcdddfa6b9b01e864b5ff727f246d6cc5e135e" gracePeriod=45
Sep 11 17:10:35.685395 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.685363    2569 generic.go:356] "Generic (PLEG): container finished" podID="6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" containerID="46460d323e9e57a085d45b309cbcdddfa6b9b01e864b5ff727f246d6cc5e135e" exitCode=0
Sep 11 17:10:35.685601 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.685430    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg" event={"ID":"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565","Type":"ContainerDied","Data":"46460d323e9e57a085d45b309cbcdddfa6b9b01e864b5ff727f246d6cc5e135e"}
Sep 11 17:10:35.753868 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.753844    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:10:35.799684 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.799654    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgrqs\" (UniqueName: \"kubernetes.io/projected/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-kube-api-access-dgrqs\") pod \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") "
Sep 11 17:10:35.799853 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.799700    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-dshm\") pod \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") "
Sep 11 17:10:35.799853 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.799736    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-home\") pod \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") "
Sep 11 17:10:35.799853 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.799764    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tmp-dir\") pod \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") "
Sep 11 17:10:35.799853 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.799832    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tls-certs\") pod \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") "
Sep 11 17:10:35.800095 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.799919    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-model-cache\") pod \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\" (UID: \"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565\") "
Sep 11 17:10:35.800095 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.800053    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-home" (OuterVolumeSpecName: "home") pod "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" (UID: "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:35.800209 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.800138    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" (UID: "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:35.800321 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.800299    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-model-cache" (OuterVolumeSpecName: "model-cache") pod "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" (UID: "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:35.800431 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.800412    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:35.800521 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.800434    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:35.800521 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.800447    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:35.802207 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.802179    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" (UID: "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:10:35.802207 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.802182    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-dshm" (OuterVolumeSpecName: "dshm") pod "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" (UID: "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:35.802384 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.802220    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-kube-api-access-dgrqs" (OuterVolumeSpecName: "kube-api-access-dgrqs") pod "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" (UID: "6bfe6f3e-2eed-4f99-9d1f-3fde6b092565"). InnerVolumeSpecName "kube-api-access-dgrqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:10:35.901270 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.901216    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgrqs\" (UniqueName: \"kubernetes.io/projected/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-kube-api-access-dgrqs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:35.901270 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.901265    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:35.901270 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:35.901276    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:36.580921 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:36.580843    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:10:36.691957 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:36.691923    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"
Sep 11 17:10:36.691957 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:36.691948    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg" event={"ID":"6bfe6f3e-2eed-4f99-9d1f-3fde6b092565","Type":"ContainerDied","Data":"825f147990e65b6452998ed3e1fed2c87c1b050242d10496db61552c83fba39f"}
Sep 11 17:10:36.692222 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:36.692002    2569 scope.go:117] "RemoveContainer" containerID="46460d323e9e57a085d45b309cbcdddfa6b9b01e864b5ff727f246d6cc5e135e"
Sep 11 17:10:36.710245 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:36.710217    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"]
Sep 11 17:10:36.715292 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:36.715267    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/auth-test-model-2fae12-kserve-75f7d84fb7-9ghcg"]
Sep 11 17:10:38.205615 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:38.205585    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" path="/var/lib/kubelet/pods/6bfe6f3e-2eed-4f99-9d1f-3fde6b092565/volumes"
Sep 11 17:10:41.289020 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:41.288981    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:41.289020 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:41.289030    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:41.301187 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:41.301156    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:41.732396 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:41.732364    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:10:42.070297 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:42.070206    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:44.676280 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:44.676247    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:44.679303 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:44.679279    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h"]
Sep 11 17:10:44.683656 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:44.683630    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2dc95c-29819100-h7l6h"]
Sep 11 17:10:44.749598 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:44.749562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:44.851516 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:44.851482    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:44.884964 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:44.884925    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"]
Sep 11 17:10:44.886876 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:44.886855    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-2dc95c-97s9b"]
Sep 11 17:10:44.950510 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:44.950424    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:45.284392 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:45.284308    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:45.384003 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:45.383973    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:46.116996 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:46.116957    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:46.205118 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:46.205046    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a57d78a-d225-4821-9a04-b5edf02b86eb" path="/var/lib/kubelet/pods/3a57d78a-d225-4821-9a04-b5edf02b86eb/volumes"
Sep 11 17:10:46.205416 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:46.205402    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6431688b-a847-47da-9aa4-5531eccf0268" path="/var/lib/kubelet/pods/6431688b-a847-47da-9aa4-5531eccf0268/volumes"
Sep 11 17:10:46.804345 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:46.804309    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:48.753965 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:48.753922    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk" podUID="44248117-a3d4-4b0c-a2f1-e640f1ec1472" containerName="main" containerID="cri-o://0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b" gracePeriod=45
Sep 11 17:10:48.763775 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:48.763738    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2" podUID="92ab41a6-a42e-4f81-976f-28317bfc1181" containerName="main" containerID="cri-o://f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c" gracePeriod=45
Sep 11 17:10:48.773545 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:48.773508    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7" podUID="b1ac9601-61bc-4d88-ad5b-6a51a52c301f" containerName="main" containerID="cri-o://316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766" gracePeriod=45
Sep 11 17:10:49.069337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.069306    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:10:49.072759 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.072731    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 17:10:49.092017 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.091935    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 17:10:49.135980 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.135939    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djqnx\" (UniqueName: \"kubernetes.io/projected/44248117-a3d4-4b0c-a2f1-e640f1ec1472-kube-api-access-djqnx\") pod \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") "
Sep 11 17:10:49.136188 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.136000    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tls-certs\") pod \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") "
Sep 11 17:10:49.136188 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.136051    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tmp-dir\") pod \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") "
Sep 11 17:10:49.136188 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.136117    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-home\") pod \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") "
Sep 11 17:10:49.136188 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.136155    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-model-cache\") pod \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") "
Sep 11 17:10:49.136407 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.136219    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-dshm\") pod \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\" (UID: \"44248117-a3d4-4b0c-a2f1-e640f1ec1472\") "
Sep 11 17:10:49.136491 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.136461    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "44248117-a3d4-4b0c-a2f1-e640f1ec1472" (UID: "44248117-a3d4-4b0c-a2f1-e640f1ec1472"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.136491 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.136483    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-home" (OuterVolumeSpecName: "home") pod "44248117-a3d4-4b0c-a2f1-e640f1ec1472" (UID: "44248117-a3d4-4b0c-a2f1-e640f1ec1472"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.136780 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.136738    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-model-cache" (OuterVolumeSpecName: "model-cache") pod "44248117-a3d4-4b0c-a2f1-e640f1ec1472" (UID: "44248117-a3d4-4b0c-a2f1-e640f1ec1472"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.137307 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.137283    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.137410 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.137311    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.137410 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.137327    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.138697 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.138664    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44248117-a3d4-4b0c-a2f1-e640f1ec1472-kube-api-access-djqnx" (OuterVolumeSpecName: "kube-api-access-djqnx") pod "44248117-a3d4-4b0c-a2f1-e640f1ec1472" (UID: "44248117-a3d4-4b0c-a2f1-e640f1ec1472"). InnerVolumeSpecName "kube-api-access-djqnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:10:49.138777 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.138746    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-dshm" (OuterVolumeSpecName: "dshm") pod "44248117-a3d4-4b0c-a2f1-e640f1ec1472" (UID: "44248117-a3d4-4b0c-a2f1-e640f1ec1472"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.138777 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.138757    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "44248117-a3d4-4b0c-a2f1-e640f1ec1472" (UID: "44248117-a3d4-4b0c-a2f1-e640f1ec1472"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:10:49.237949 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.237908    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-home\") pod \"92ab41a6-a42e-4f81-976f-28317bfc1181\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") "
Sep 11 17:10:49.237949 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.237952    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ab41a6-a42e-4f81-976f-28317bfc1181-tls-certs\") pod \"92ab41a6-a42e-4f81-976f-28317bfc1181\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") "
Sep 11 17:10:49.238290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.237972    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-dshm\") pod \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") "
Sep 11 17:10:49.238290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238016    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-dshm\") pod \"92ab41a6-a42e-4f81-976f-28317bfc1181\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") "
Sep 11 17:10:49.238290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238036    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clv45\" (UniqueName: \"kubernetes.io/projected/92ab41a6-a42e-4f81-976f-28317bfc1181-kube-api-access-clv45\") pod \"92ab41a6-a42e-4f81-976f-28317bfc1181\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") "
Sep 11 17:10:49.238290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238060    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tls-certs\") pod \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") "
Sep 11 17:10:49.238290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238106    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tmp-dir\") pod \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") "
Sep 11 17:10:49.238290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238131    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-tmp-dir\") pod \"92ab41a6-a42e-4f81-976f-28317bfc1181\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") "
Sep 11 17:10:49.238290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238269    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-home" (OuterVolumeSpecName: "home") pod "92ab41a6-a42e-4f81-976f-28317bfc1181" (UID: "92ab41a6-a42e-4f81-976f-28317bfc1181"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.238659 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238329    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-model-cache\") pod \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") "
Sep 11 17:10:49.238659 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238428    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "92ab41a6-a42e-4f81-976f-28317bfc1181" (UID: "92ab41a6-a42e-4f81-976f-28317bfc1181"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.238659 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238436    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-home\") pod \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") "
Sep 11 17:10:49.238659 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238493    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b1ac9601-61bc-4d88-ad5b-6a51a52c301f" (UID: "b1ac9601-61bc-4d88-ad5b-6a51a52c301f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.238659 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238581    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh8tt\" (UniqueName: \"kubernetes.io/projected/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-kube-api-access-bh8tt\") pod \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\" (UID: \"b1ac9601-61bc-4d88-ad5b-6a51a52c301f\") "
Sep 11 17:10:49.238659 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238615    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-home" (OuterVolumeSpecName: "home") pod "b1ac9601-61bc-4d88-ad5b-6a51a52c301f" (UID: "b1ac9601-61bc-4d88-ad5b-6a51a52c301f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.238659 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238624    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-model-cache\") pod \"92ab41a6-a42e-4f81-976f-28317bfc1181\" (UID: \"92ab41a6-a42e-4f81-976f-28317bfc1181\") "
Sep 11 17:10:49.239024 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238708    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-model-cache" (OuterVolumeSpecName: "model-cache") pod "b1ac9601-61bc-4d88-ad5b-6a51a52c301f" (UID: "b1ac9601-61bc-4d88-ad5b-6a51a52c301f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.239024 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238971    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.239024 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.238990    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.239024 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.239004    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.239024 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.239019    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44248117-a3d4-4b0c-a2f1-e640f1ec1472-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.239318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.239031    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.239318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.239044    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.239318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.239058    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djqnx\" (UniqueName: \"kubernetes.io/projected/44248117-a3d4-4b0c-a2f1-e640f1ec1472-kube-api-access-djqnx\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.239318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.239102    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44248117-a3d4-4b0c-a2f1-e640f1ec1472-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.239318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.239170    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-model-cache" (OuterVolumeSpecName: "model-cache") pod "92ab41a6-a42e-4f81-976f-28317bfc1181" (UID: "92ab41a6-a42e-4f81-976f-28317bfc1181"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.240548 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.240520    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ab41a6-a42e-4f81-976f-28317bfc1181-kube-api-access-clv45" (OuterVolumeSpecName: "kube-api-access-clv45") pod "92ab41a6-a42e-4f81-976f-28317bfc1181" (UID: "92ab41a6-a42e-4f81-976f-28317bfc1181"). InnerVolumeSpecName "kube-api-access-clv45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:10:49.240844 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.240817    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b1ac9601-61bc-4d88-ad5b-6a51a52c301f" (UID: "b1ac9601-61bc-4d88-ad5b-6a51a52c301f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:10:49.240936 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.240817    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92ab41a6-a42e-4f81-976f-28317bfc1181-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "92ab41a6-a42e-4f81-976f-28317bfc1181" (UID: "92ab41a6-a42e-4f81-976f-28317bfc1181"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:10:49.240936 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.240820    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-dshm" (OuterVolumeSpecName: "dshm") pod "92ab41a6-a42e-4f81-976f-28317bfc1181" (UID: "92ab41a6-a42e-4f81-976f-28317bfc1181"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.240936 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.240905    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-kube-api-access-bh8tt" (OuterVolumeSpecName: "kube-api-access-bh8tt") pod "b1ac9601-61bc-4d88-ad5b-6a51a52c301f" (UID: "b1ac9601-61bc-4d88-ad5b-6a51a52c301f"). InnerVolumeSpecName "kube-api-access-bh8tt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:10:49.241038 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.240951    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-dshm" (OuterVolumeSpecName: "dshm") pod "b1ac9601-61bc-4d88-ad5b-6a51a52c301f" (UID: "b1ac9601-61bc-4d88-ad5b-6a51a52c301f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:49.340344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.340242    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh8tt\" (UniqueName: \"kubernetes.io/projected/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-kube-api-access-bh8tt\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.340344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.340278    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.340344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.340291    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ab41a6-a42e-4f81-976f-28317bfc1181-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.340344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.340302    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.340344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.340312    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ab41a6-a42e-4f81-976f-28317bfc1181-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.340344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.340323    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clv45\" (UniqueName: \"kubernetes.io/projected/92ab41a6-a42e-4f81-976f-28317bfc1181-kube-api-access-clv45\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.340344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.340334    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1ac9601-61bc-4d88-ad5b-6a51a52c301f-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:49.766523 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.766488    2569 generic.go:356] "Generic (PLEG): container finished" podID="44248117-a3d4-4b0c-a2f1-e640f1ec1472" containerID="0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b" exitCode=0
Sep 11 17:10:49.767045 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.766536    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"
Sep 11 17:10:49.767045 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.766580    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk" event={"ID":"44248117-a3d4-4b0c-a2f1-e640f1ec1472","Type":"ContainerDied","Data":"0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b"}
Sep 11 17:10:49.767045 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.766625    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk" event={"ID":"44248117-a3d4-4b0c-a2f1-e640f1ec1472","Type":"ContainerDied","Data":"62c3b495f05a8c38326f2d17c32450a392cbe5f33736353d6bb279888ca77558"}
Sep 11 17:10:49.767045 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.766643    2569 scope.go:117] "RemoveContainer" containerID="0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b"
Sep 11 17:10:49.768352 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.768327    2569 generic.go:356] "Generic (PLEG): container finished" podID="b1ac9601-61bc-4d88-ad5b-6a51a52c301f" containerID="316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766" exitCode=0
Sep 11 17:10:49.768447 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.768375    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"
Sep 11 17:10:49.768447 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.768418    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7" event={"ID":"b1ac9601-61bc-4d88-ad5b-6a51a52c301f","Type":"ContainerDied","Data":"316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766"}
Sep 11 17:10:49.768558 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.768448    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7" event={"ID":"b1ac9601-61bc-4d88-ad5b-6a51a52c301f","Type":"ContainerDied","Data":"212f5117a3fe62638b5f06eed2b80c2b44d895bf7f6e8e6a7322450a5340ad28"}
Sep 11 17:10:49.770292 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.770063    2569 generic.go:356] "Generic (PLEG): container finished" podID="92ab41a6-a42e-4f81-976f-28317bfc1181" containerID="f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c" exitCode=0
Sep 11 17:10:49.770292 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.770138    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"
Sep 11 17:10:49.770292 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.770155    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2" event={"ID":"92ab41a6-a42e-4f81-976f-28317bfc1181","Type":"ContainerDied","Data":"f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c"}
Sep 11 17:10:49.770292 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.770183    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2" event={"ID":"92ab41a6-a42e-4f81-976f-28317bfc1181","Type":"ContainerDied","Data":"516759d823f8abfd42daf57739b1ed702296a84ec3cca017dec094812ba34b9b"}
Sep 11 17:10:49.787679 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.787655    2569 scope.go:117] "RemoveContainer" containerID="0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b"
Sep 11 17:10:49.788032 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:10:49.788011    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b\": container with ID starting with 0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b not found: ID does not exist" containerID="0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b"
Sep 11 17:10:49.788151 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.788047    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b"} err="failed to get container status \"0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b\": rpc error: code = NotFound desc = could not find container \"0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b\": container with ID starting with 0b1e13ab098a95eb13628c8008bb3b10fe89b5fc99478db04ebf9b4d7360d27b not found: ID does not exist"
Sep 11 17:10:49.788151 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.788094    2569 scope.go:117] "RemoveContainer" containerID="316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766"
Sep 11 17:10:49.796780 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.796760    2569 scope.go:117] "RemoveContainer" containerID="316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766"
Sep 11 17:10:49.797102 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:10:49.797069    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766\": container with ID starting with 316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766 not found: ID does not exist" containerID="316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766"
Sep 11 17:10:49.797179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.797110    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766"} err="failed to get container status \"316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766\": rpc error: code = NotFound desc = could not find container \"316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766\": container with ID starting with 316c4eca365aabdda9451c01b586d10d285543a62a172c42e8ff883436ed1766 not found: ID does not exist"
Sep 11 17:10:49.797179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.797128    2569 scope.go:117] "RemoveContainer" containerID="f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c"
Sep 11 17:10:49.804141 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.804051    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"]
Sep 11 17:10:49.806997 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.806976    2569 scope.go:117] "RemoveContainer" containerID="f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c"
Sep 11 17:10:49.807407 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:10:49.807387    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c\": container with ID starting with f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c not found: ID does not exist" containerID="f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c"
Sep 11 17:10:49.807471 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.807416    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c"} err="failed to get container status \"f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c\": rpc error: code = NotFound desc = could not find container \"f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c\": container with ID starting with f5a1505d9fea80a78045f58378880ffb202d7a221448ae75ff856fa85b117f2c not found: ID does not exist"
Sep 11 17:10:49.807548 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.807530    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/simulator-w4-2dc95c-kserve-54755b99c5-jd9b7"]
Sep 11 17:10:49.817273 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.817242    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"]
Sep 11 17:10:49.823460 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.823431    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/distinct-2-w4-2dc95c-kserve-bbc9bff54-gg2zk"]
Sep 11 17:10:49.832613 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.832584    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"]
Sep 11 17:10:49.836222 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:49.836195    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-2dc95c/premium-w4-2dc95c-kserve-7ddc5c6b58-tr9g2"]
Sep 11 17:10:50.205583 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:50.205550    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44248117-a3d4-4b0c-a2f1-e640f1ec1472" path="/var/lib/kubelet/pods/44248117-a3d4-4b0c-a2f1-e640f1ec1472/volumes"
Sep 11 17:10:50.205962 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:50.205948    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92ab41a6-a42e-4f81-976f-28317bfc1181" path="/var/lib/kubelet/pods/92ab41a6-a42e-4f81-976f-28317bfc1181/volumes"
Sep 11 17:10:50.206343 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:50.206330    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1ac9601-61bc-4d88-ad5b-6a51a52c301f" path="/var/lib/kubelet/pods/b1ac9601-61bc-4d88-ad5b-6a51a52c301f/volumes"
Sep 11 17:10:52.113194 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:52.113154    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:52.211640 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:52.211607    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:52.395334 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:52.395301    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"]
Sep 11 17:10:52.395583 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:52.395550    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy" containerID="cri-o://ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6" gracePeriod=30
Sep 11 17:10:52.416570 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:52.416543    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:10:57.645344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.645316    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 17:10:57.822831 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.822737    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-credential-socket\") pod \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") "
Sep 11 17:10:57.822831 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.822784    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-podinfo\") pod \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") "
Sep 11 17:10:57.822831 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.822813    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-data\") pod \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") "
Sep 11 17:10:57.823156 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.822841    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-envoy\") pod \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") "
Sep 11 17:10:57.823156 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.822899    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-socket\") pod \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") "
Sep 11 17:10:57.823156 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.822981    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlpp7\" (UniqueName: \"kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-kube-api-access-nlpp7\") pod \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") "
Sep 11 17:10:57.823156 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823019    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3bf67c2-4778-4df3-acc5-e64b3282c37f" (UID: "d3bf67c2-4778-4df3-acc5-e64b3282c37f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:57.823156 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823037    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-certs\") pod \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") "
Sep 11 17:10:57.823156 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823121    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istiod-ca-cert\") pod \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") "
Sep 11 17:10:57.823156 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823149    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3bf67c2-4778-4df3-acc5-e64b3282c37f" (UID: "d3bf67c2-4778-4df3-acc5-e64b3282c37f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:57.823516 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823170    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-token\") pod \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\" (UID: \"d3bf67c2-4778-4df3-acc5-e64b3282c37f\") "
Sep 11 17:10:57.823516 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823412    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3bf67c2-4778-4df3-acc5-e64b3282c37f" (UID: "d3bf67c2-4778-4df3-acc5-e64b3282c37f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:57.823516 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823464    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3bf67c2-4778-4df3-acc5-e64b3282c37f" (UID: "d3bf67c2-4778-4df3-acc5-e64b3282c37f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:57.823682 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823568    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3bf67c2-4778-4df3-acc5-e64b3282c37f" (UID: "d3bf67c2-4778-4df3-acc5-e64b3282c37f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:10:57.823682 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823587    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:57.823682 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823614    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:57.823682 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823629    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:57.823682 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.823642    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:57.825488 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.825457    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3bf67c2-4778-4df3-acc5-e64b3282c37f" (UID: "d3bf67c2-4778-4df3-acc5-e64b3282c37f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:10:57.825602 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.825506    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3bf67c2-4778-4df3-acc5-e64b3282c37f" (UID: "d3bf67c2-4778-4df3-acc5-e64b3282c37f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:10:57.825772 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.825751    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3bf67c2-4778-4df3-acc5-e64b3282c37f" (UID: "d3bf67c2-4778-4df3-acc5-e64b3282c37f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:10:57.825815 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.825772    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-kube-api-access-nlpp7" (OuterVolumeSpecName: "kube-api-access-nlpp7") pod "d3bf67c2-4778-4df3-acc5-e64b3282c37f" (UID: "d3bf67c2-4778-4df3-acc5-e64b3282c37f"). InnerVolumeSpecName "kube-api-access-nlpp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:10:57.832411 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.832381    2569 generic.go:356] "Generic (PLEG): container finished" podID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerID="ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6" exitCode=0
Sep 11 17:10:57.832515 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.832423    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"
Sep 11 17:10:57.832515 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.832454    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" event={"ID":"d3bf67c2-4778-4df3-acc5-e64b3282c37f","Type":"ContainerDied","Data":"ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6"}
Sep 11 17:10:57.832515 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.832507    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x" event={"ID":"d3bf67c2-4778-4df3-acc5-e64b3282c37f","Type":"ContainerDied","Data":"5bd2b6232c3ef692f159101fa02e87f48508dbffa69d23571333121141e4a272"}
Sep 11 17:10:57.832706 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.832532    2569 scope.go:117] "RemoveContainer" containerID="ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6"
Sep 11 17:10:57.846659 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.846642    2569 scope.go:117] "RemoveContainer" containerID="703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15"
Sep 11 17:10:57.854642 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.854616    2569 scope.go:117] "RemoveContainer" containerID="ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6"
Sep 11 17:10:57.855238 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:10:57.855209    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6\": container with ID starting with ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6 not found: ID does not exist" containerID="ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6"
Sep 11 17:10:57.855415 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.855268    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6"} err="failed to get container status \"ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6\": rpc error: code = NotFound desc = could not find container \"ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6\": container with ID starting with ead6d9e1a143603fd0b900bc4d020db6a743a34451af0e840c173180ff6b70c6 not found: ID does not exist"
Sep 11 17:10:57.855415 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.855295    2569 scope.go:117] "RemoveContainer" containerID="703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15"
Sep 11 17:10:57.855659 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:10:57.855633    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15\": container with ID starting with 703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15 not found: ID does not exist" containerID="703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15"
Sep 11 17:10:57.855659 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.855666    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15"} err="failed to get container status \"703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15\": rpc error: code = NotFound desc = could not find container \"703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15\": container with ID starting with 703ef3f4a15d44d9ce03336f1cdf4d9bb6ab73645845ff8f496a324d949cea15 not found: ID does not exist"
Sep 11 17:10:57.863506 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.863479    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"]
Sep 11 17:10:57.863674 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.863661    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-2dc95c-openshift-default-7f7c49787b-kbn8x"]
Sep 11 17:10:57.925160 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.925122    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:57.925160 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.925153    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:57.925160 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.925162    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:57.925160 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.925170    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3bf67c2-4778-4df3-acc5-e64b3282c37f-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:57.925425 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:57.925179    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlpp7\" (UniqueName: \"kubernetes.io/projected/d3bf67c2-4778-4df3-acc5-e64b3282c37f-kube-api-access-nlpp7\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:10:58.205288 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:58.205253    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" path="/var/lib/kubelet/pods/d3bf67c2-4778-4df3-acc5-e64b3282c37f/volumes"
Sep 11 17:10:58.984443 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:10:58.984407    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:11:10.045103 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:10.045044    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:11:10.114529 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:10.114497    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:11:11.316476 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:11.316445    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:11:18.286697 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.286657    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"]
Sep 11 17:11:18.287249 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287229    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ac9601-61bc-4d88-ad5b-6a51a52c301f" containerName="main"
Sep 11 17:11:18.287337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287251    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ac9601-61bc-4d88-ad5b-6a51a52c301f" containerName="main"
Sep 11 17:11:18.287337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287264    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" containerName="main"
Sep 11 17:11:18.287337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287271    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" containerName="main"
Sep 11 17:11:18.287337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287288    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy"
Sep 11 17:11:18.287337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287298    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy"
Sep 11 17:11:18.287337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287309    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92ab41a6-a42e-4f81-976f-28317bfc1181" containerName="main"
Sep 11 17:11:18.287337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287318    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="92ab41a6-a42e-4f81-976f-28317bfc1181" containerName="main"
Sep 11 17:11:18.287337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287339    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44248117-a3d4-4b0c-a2f1-e640f1ec1472" containerName="main"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287346    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="44248117-a3d4-4b0c-a2f1-e640f1ec1472" containerName="main"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287355    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6431688b-a847-47da-9aa4-5531eccf0268" containerName="revoke-keys"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287363    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6431688b-a847-47da-9aa4-5531eccf0268" containerName="revoke-keys"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287384    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287392    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287516    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1ac9601-61bc-4d88-ad5b-6a51a52c301f" containerName="main"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287526    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="92ab41a6-a42e-4f81-976f-28317bfc1181" containerName="main"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287537    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bfe6f3e-2eed-4f99-9d1f-3fde6b092565" containerName="main"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287547    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287555    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="44248117-a3d4-4b0c-a2f1-e640f1ec1472" containerName="main"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287565    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3bf67c2-4778-4df3-acc5-e64b3282c37f" containerName="istio-proxy"
Sep 11 17:11:18.287717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.287576    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6431688b-a847-47da-9aa4-5531eccf0268" containerName="revoke-keys"
Sep 11 17:11:18.292638 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.292617    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.295949 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.295930    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-327a3a\"/\"kube-root-ca.crt\""
Sep 11 17:11:18.296064 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.295990    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-327a3a\"/\"openshift-service-ca.crt\""
Sep 11 17:11:18.296064 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.295998    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-327a3a\"/\"sub-test-model-327a3a-kserve-self-signed-certs\""
Sep 11 17:11:18.296064 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.295998    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-327a3a\"/\"default-dockercfg-5rzw7\""
Sep 11 17:11:18.301811 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.301787    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"]
Sep 11 17:11:18.334339 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.334311    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:11:18.421617 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.421575    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tls-certs\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.421779 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.421630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-home\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.421779 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.421686    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-dshm\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.421779 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.421769    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvhr2\" (UniqueName: \"kubernetes.io/projected/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-kube-api-access-zvhr2\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.421891 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.421799    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tmp-dir\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.421891 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.421822    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-model-cache\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.522733 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.522689    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvhr2\" (UniqueName: \"kubernetes.io/projected/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-kube-api-access-zvhr2\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.522733 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.522738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tmp-dir\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.522967 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.522854    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-model-cache\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.522967 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.522944    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tls-certs\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.523105 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.522994    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-home\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.523105 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.523035    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-dshm\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.523287 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.523269    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tmp-dir\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.523368 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.523348    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-model-cache\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.523428 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.523358    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-home\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.525352 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.525331    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-dshm\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.525510 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.525492    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tls-certs\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.531267 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.531244    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvhr2\" (UniqueName: \"kubernetes.io/projected/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-kube-api-access-zvhr2\") pod \"sub-test-model-327a3a-kserve-5dc88c947c-btf75\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.603101 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.602991    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:18.875401 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.875378    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"]
Sep 11 17:11:18.877363 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:11:18.877337    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72a35d04_cbcf_4cf2_a8da_cbe3735bbdc8.slice/crio-ac336b8e4afcdf25d07ac4d455b0a30ed234cb66a2c024023d6d0f00e2346f3d WatchSource:0}: Error finding container ac336b8e4afcdf25d07ac4d455b0a30ed234cb66a2c024023d6d0f00e2346f3d: Status 404 returned error can't find the container with id ac336b8e4afcdf25d07ac4d455b0a30ed234cb66a2c024023d6d0f00e2346f3d
Sep 11 17:11:18.950671 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.950635    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75" event={"ID":"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8","Type":"ContainerStarted","Data":"46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4"}
Sep 11 17:11:18.950797 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.950684    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75" event={"ID":"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8","Type":"ContainerStarted","Data":"ac336b8e4afcdf25d07ac4d455b0a30ed234cb66a2c024023d6d0f00e2346f3d"}
Sep 11 17:11:18.967893 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:18.967845    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75" podStartSLOduration=0.967827377 podStartE2EDuration="967.827377ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:11:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:11:18.967070881 +0000 UTC m=+1675.374289555" watchObservedRunningTime="2026-09-11 17:11:18.967827377 +0000 UTC m=+1675.375046024"
Sep 11 17:11:20.434149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:20.434118    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:11:20.534577 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:20.534524    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:11:22.279494 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:22.279459    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:11:24.404963 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:24.404933    2569 scope.go:117] "RemoveContainer" containerID="f5576b066ed43103e31b6b0627a9c35d63729f6ce907a992a6aeb92eba769e22"
Sep 11 17:11:24.413482 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:24.413460    2569 scope.go:117] "RemoveContainer" containerID="2f6b734a397a9facc8b8c1ee4dc5f765a5ab6f783ce03f03db0933d708f55f4b"
Sep 11 17:11:26.628934 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:26.628904    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:11:28.603142 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:28.603071    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:28.603558 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:28.603163    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:28.615784 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:28.615755    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:29.012737 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:29.012709    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:11:41.076261 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:41.076221    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:11:54.087263 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:54.087175    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:11:56.096739 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:56.096708    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7951-06df0e"]
Sep 11 17:11:56.100221 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:56.100203    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7951-06df0e"
Sep 11 17:11:56.102289 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:56.102259    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-t2nxk\""
Sep 11 17:11:56.106254 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:56.106230    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7951-06df0e"]
Sep 11 17:11:56.189664 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:56.189629    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9dzs\" (UniqueName: \"kubernetes.io/projected/04ca28ce-cb5c-4696-b8fd-852fea95c799-kube-api-access-x9dzs\") pod \"test-curl-7951-06df0e\" (UID: \"04ca28ce-cb5c-4696-b8fd-852fea95c799\") " pod="opendatahub/test-curl-7951-06df0e"
Sep 11 17:11:56.290250 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:56.290210    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9dzs\" (UniqueName: \"kubernetes.io/projected/04ca28ce-cb5c-4696-b8fd-852fea95c799-kube-api-access-x9dzs\") pod \"test-curl-7951-06df0e\" (UID: \"04ca28ce-cb5c-4696-b8fd-852fea95c799\") " pod="opendatahub/test-curl-7951-06df0e"
Sep 11 17:11:56.299308 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:56.299276    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9dzs\" (UniqueName: \"kubernetes.io/projected/04ca28ce-cb5c-4696-b8fd-852fea95c799-kube-api-access-x9dzs\") pod \"test-curl-7951-06df0e\" (UID: \"04ca28ce-cb5c-4696-b8fd-852fea95c799\") " pod="opendatahub/test-curl-7951-06df0e"
Sep 11 17:11:56.410753 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:56.410713    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7951-06df0e"
Sep 11 17:11:56.683558 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:56.683527    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7951-06df0e"]
Sep 11 17:11:56.685755 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:11:56.685724    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04ca28ce_cb5c_4696_b8fd_852fea95c799.slice/crio-30f1c5ceb68443e361257b6ab8f05120f206130d7e476c0f3a4d8ae586d0b18b WatchSource:0}: Error finding container 30f1c5ceb68443e361257b6ab8f05120f206130d7e476c0f3a4d8ae586d0b18b: Status 404 returned error can't find the container with id 30f1c5ceb68443e361257b6ab8f05120f206130d7e476c0f3a4d8ae586d0b18b
Sep 11 17:11:56.687623 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:56.687605    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:11:57.141876 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:57.141837    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7951-06df0e" event={"ID":"04ca28ce-cb5c-4696-b8fd-852fea95c799","Type":"ContainerStarted","Data":"30f1c5ceb68443e361257b6ab8f05120f206130d7e476c0f3a4d8ae586d0b18b"}
Sep 11 17:11:58.155560 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:58.155524    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7951-06df0e" event={"ID":"04ca28ce-cb5c-4696-b8fd-852fea95c799","Type":"ContainerStarted","Data":"fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5"}
Sep 11 17:11:58.175022 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:58.174962    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7951-06df0e" podStartSLOduration=1.153736435 podStartE2EDuration="2.174942075s" totalImagesPullingTime="1.02120564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:11:58.174228307 +0000 UTC m=+1714.581446950" watchObservedRunningTime="2026-09-11 17:11:58.174942075 +0000 UTC m=+1714.582160717"
Sep 11 17:11:58.870594 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:58.870554    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7951-06df0e"]
Sep 11 17:11:58.874614 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:58.874583    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7951-06df0e"]
Sep 11 17:11:59.012894 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:59.012813    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7951-b7d640"]
Sep 11 17:11:59.013277 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:59.013260    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ca28ce-cb5c-4696-b8fd-852fea95c799" containerName="test-curl-7951-06df0e"
Sep 11 17:11:59.013328 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:59.013279    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ca28ce-cb5c-4696-b8fd-852fea95c799" containerName="test-curl-7951-06df0e"
Sep 11 17:11:59.013364 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:59.013339    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="04ca28ce-cb5c-4696-b8fd-852fea95c799" containerName="test-curl-7951-06df0e"
Sep 11 17:11:59.016449 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:59.016430    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7951-b7d640"
Sep 11 17:11:59.021354 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:59.021334    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7951-b7d640"]
Sep 11 17:11:59.121180 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:59.121143    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjx6v\" (UniqueName: \"kubernetes.io/projected/d4a30b98-4c3f-4796-85a9-5fba827898d9-kube-api-access-pjx6v\") pod \"test-curl-7951-b7d640\" (UID: \"d4a30b98-4c3f-4796-85a9-5fba827898d9\") " pod="opendatahub/test-curl-7951-b7d640"
Sep 11 17:11:59.160514 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:11:59.160487    2569 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7951-06df0e" container="test-curl-7951-06df0e"
Sep 11 17:11:59.222204 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:59.222167    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjx6v\" (UniqueName: \"kubernetes.io/projected/d4a30b98-4c3f-4796-85a9-5fba827898d9-kube-api-access-pjx6v\") pod \"test-curl-7951-b7d640\" (UID: \"d4a30b98-4c3f-4796-85a9-5fba827898d9\") " pod="opendatahub/test-curl-7951-b7d640"
Sep 11 17:11:59.233429 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:59.233391    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjx6v\" (UniqueName: \"kubernetes.io/projected/d4a30b98-4c3f-4796-85a9-5fba827898d9-kube-api-access-pjx6v\") pod \"test-curl-7951-b7d640\" (UID: \"d4a30b98-4c3f-4796-85a9-5fba827898d9\") " pod="opendatahub/test-curl-7951-b7d640"
Sep 11 17:11:59.327477 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:59.327389    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7951-b7d640"
Sep 11 17:11:59.595837 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:11:59.595748    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7951-b7d640"]
Sep 11 17:11:59.598066 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:11:59.598039    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4a30b98_4c3f_4796_85a9_5fba827898d9.slice/crio-1b6e7b0eca82ddaa1b1d26aa0efe1e71645d6f7b0a7d15b0c651981c3381823a WatchSource:0}: Error finding container 1b6e7b0eca82ddaa1b1d26aa0efe1e71645d6f7b0a7d15b0c651981c3381823a: Status 404 returned error can't find the container with id 1b6e7b0eca82ddaa1b1d26aa0efe1e71645d6f7b0a7d15b0c651981c3381823a
Sep 11 17:12:00.167157 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:00.167110    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7951-b7d640" event={"ID":"d4a30b98-4c3f-4796-85a9-5fba827898d9","Type":"ContainerStarted","Data":"c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f"}
Sep 11 17:12:00.167661 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:00.167166    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7951-b7d640" event={"ID":"d4a30b98-4c3f-4796-85a9-5fba827898d9","Type":"ContainerStarted","Data":"1b6e7b0eca82ddaa1b1d26aa0efe1e71645d6f7b0a7d15b0c651981c3381823a"}
Sep 11 17:12:00.167661 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:00.167319    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7951-06df0e" podUID="04ca28ce-cb5c-4696-b8fd-852fea95c799" containerName="test-curl-7951-06df0e" containerID="cri-o://fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5" gracePeriod=2
Sep 11 17:12:00.181216 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:00.181161    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7951-b7d640" podStartSLOduration=2.181145067 podStartE2EDuration="2.181145067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:11:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:12:00.179973922 +0000 UTC m=+1716.587192574" watchObservedRunningTime="2026-09-11 17:12:00.181145067 +0000 UTC m=+1716.588363714"
Sep 11 17:12:00.181529 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:00.181501    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7951-06df0e\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="04ca28ce-cb5c-4696-b8fd-852fea95c799" pod="opendatahub/test-curl-7951-06df0e"
Sep 11 17:12:00.809566 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:00.809529    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7951-b7d640"]
Sep 11 17:12:00.814902 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:00.814871    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7951-b7d640"]
Sep 11 17:12:01.173026 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:01.172997    2569 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7951-b7d640" container="test-curl-7951-b7d640"
Sep 11 17:12:02.179454 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:02.179418    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7951-b7d640" podUID="d4a30b98-4c3f-4796-85a9-5fba827898d9" containerName="test-curl-7951-b7d640" containerID="cri-o://c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f" gracePeriod=2
Sep 11 17:12:02.181831 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:02.181732    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7951-b7d640\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="d4a30b98-4c3f-4796-85a9-5fba827898d9" pod="opendatahub/test-curl-7951-b7d640"
Sep 11 17:12:02.350060 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:02.349993    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7951-06df0e"
Sep 11 17:12:02.452893 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:02.452858    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9dzs\" (UniqueName: \"kubernetes.io/projected/04ca28ce-cb5c-4696-b8fd-852fea95c799-kube-api-access-x9dzs\") pod \"04ca28ce-cb5c-4696-b8fd-852fea95c799\" (UID: \"04ca28ce-cb5c-4696-b8fd-852fea95c799\") "
Sep 11 17:12:02.455220 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:02.455182    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ca28ce-cb5c-4696-b8fd-852fea95c799-kube-api-access-x9dzs" (OuterVolumeSpecName: "kube-api-access-x9dzs") pod "04ca28ce-cb5c-4696-b8fd-852fea95c799" (UID: "04ca28ce-cb5c-4696-b8fd-852fea95c799"). InnerVolumeSpecName "kube-api-access-x9dzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:12:02.553683 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:02.553644    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9dzs\" (UniqueName: \"kubernetes.io/projected/04ca28ce-cb5c-4696-b8fd-852fea95c799-kube-api-access-x9dzs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:02.959511 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:02.959473    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"]
Sep 11 17:12:02.980785 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:02.980752    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:12:03.186336 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:03.186297    2569 generic.go:356] "Generic (PLEG): container finished" podID="04ca28ce-cb5c-4696-b8fd-852fea95c799" containerID="fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5" exitCode=137
Sep 11 17:12:03.186336 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:03.186337    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7951-06df0e"
Sep 11 17:12:03.186794 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:03.186410    2569 scope.go:117] "RemoveContainer" containerID="fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5"
Sep 11 17:12:03.195816 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:03.195798    2569 scope.go:117] "RemoveContainer" containerID="fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5"
Sep 11 17:12:03.196184 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:03.196159    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5\": container with ID starting with fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5 not found: ID does not exist" containerID="fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5"
Sep 11 17:12:03.196272 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:03.196193    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5"} err="failed to get container status \"fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5\": rpc error: code = NotFound desc = could not find container \"fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5\": container with ID starting with fdd88a80022909c426fdb4227d545cbe8a66fdb2092ab4b2060689c91a6ec8a5 not found: ID does not exist"
Sep 11 17:12:03.196787 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:03.196758    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7951-06df0e\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="04ca28ce-cb5c-4696-b8fd-852fea95c799" pod="opendatahub/test-curl-7951-06df0e"
Sep 11 17:12:03.992747 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:03.992713    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"]
Sep 11 17:12:04.011404 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:04.011358    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7951-06df0e\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="04ca28ce-cb5c-4696-b8fd-852fea95c799" pod="opendatahub/test-curl-7951-06df0e"
Sep 11 17:12:04.014317 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:04.014284    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:12:04.203509 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:04.203471    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7951-06df0e\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="04ca28ce-cb5c-4696-b8fd-852fea95c799" pod="opendatahub/test-curl-7951-06df0e"
Sep 11 17:12:04.205473 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:04.205449    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04ca28ce-cb5c-4696-b8fd-852fea95c799" path="/var/lib/kubelet/pods/04ca28ce-cb5c-4696-b8fd-852fea95c799/volumes"
Sep 11 17:12:04.363341 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:04.363317    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7951-b7d640"
Sep 11 17:12:04.365211 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:04.365187    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7951-b7d640\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="d4a30b98-4c3f-4796-85a9-5fba827898d9" pod="opendatahub/test-curl-7951-b7d640"
Sep 11 17:12:04.472242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:04.472198    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjx6v\" (UniqueName: \"kubernetes.io/projected/d4a30b98-4c3f-4796-85a9-5fba827898d9-kube-api-access-pjx6v\") pod \"d4a30b98-4c3f-4796-85a9-5fba827898d9\" (UID: \"d4a30b98-4c3f-4796-85a9-5fba827898d9\") "
Sep 11 17:12:04.474361 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:04.474333    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a30b98-4c3f-4796-85a9-5fba827898d9-kube-api-access-pjx6v" (OuterVolumeSpecName: "kube-api-access-pjx6v") pod "d4a30b98-4c3f-4796-85a9-5fba827898d9" (UID: "d4a30b98-4c3f-4796-85a9-5fba827898d9"). InnerVolumeSpecName "kube-api-access-pjx6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:12:04.574048 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:04.573942    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjx6v\" (UniqueName: \"kubernetes.io/projected/d4a30b98-4c3f-4796-85a9-5fba827898d9-kube-api-access-pjx6v\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:05.180589 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:05.180555    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:12:05.201451 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:05.201420    2569 generic.go:356] "Generic (PLEG): container finished" podID="d4a30b98-4c3f-4796-85a9-5fba827898d9" containerID="c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f" exitCode=137
Sep 11 17:12:05.201451 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:05.201446    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7951-b7d640"
Sep 11 17:12:05.201675 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:05.201508    2569 scope.go:117] "RemoveContainer" containerID="c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f"
Sep 11 17:12:05.203389 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:05.203359    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7951-b7d640\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="d4a30b98-4c3f-4796-85a9-5fba827898d9" pod="opendatahub/test-curl-7951-b7d640"
Sep 11 17:12:05.211052 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:05.211033    2569 scope.go:117] "RemoveContainer" containerID="c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f"
Sep 11 17:12:05.211370 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:05.211352    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f\": container with ID starting with c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f not found: ID does not exist" containerID="c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f"
Sep 11 17:12:05.211413 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:05.211381    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f"} err="failed to get container status \"c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f\": rpc error: code = NotFound desc = could not find container \"c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f\": container with ID starting with c61e9ac0010672ff1dd2b341a6f144f27145d88d84c9865561251a2f92b5eb5f not found: ID does not exist"
Sep 11 17:12:05.211757 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:05.211738    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7951-b7d640\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="d4a30b98-4c3f-4796-85a9-5fba827898d9" pod="opendatahub/test-curl-7951-b7d640"
Sep 11 17:12:06.207343 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:06.207300    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4a30b98-4c3f-4796-85a9-5fba827898d9" path="/var/lib/kubelet/pods/d4a30b98-4c3f-4796-85a9-5fba827898d9/volumes"
Sep 11 17:12:11.076532 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.076488    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"]
Sep 11 17:12:11.076956 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.076908    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a30b98-4c3f-4796-85a9-5fba827898d9" containerName="test-curl-7951-b7d640"
Sep 11 17:12:11.076956 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.076919    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a30b98-4c3f-4796-85a9-5fba827898d9" containerName="test-curl-7951-b7d640"
Sep 11 17:12:11.077113 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.076984    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a30b98-4c3f-4796-85a9-5fba827898d9" containerName="test-curl-7951-b7d640"
Sep 11 17:12:11.081462 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.081427    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.083534 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.083506    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2fae12\"/\"rate-test-model-2fae12-kserve-self-signed-certs\""
Sep 11 17:12:11.089770 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.089745    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"]
Sep 11 17:12:11.101102 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.101037    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:12:11.240141 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.240103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a215a0c4-42fb-4b21-b6ce-c610120fff14-tls-certs\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.240141 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.240142    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-home\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.240351 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.240182    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-dshm\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.240351 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.240224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-model-cache\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.240351 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.240264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8mj4\" (UniqueName: \"kubernetes.io/projected/a215a0c4-42fb-4b21-b6ce-c610120fff14-kube-api-access-c8mj4\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.240351 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.240321    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-tmp-dir\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.341749 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.341655    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a215a0c4-42fb-4b21-b6ce-c610120fff14-tls-certs\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.341749 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.341700    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-home\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.341749 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.341727    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-dshm\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.341978 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.341755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-model-cache\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.341978 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.341816    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8mj4\" (UniqueName: \"kubernetes.io/projected/a215a0c4-42fb-4b21-b6ce-c610120fff14-kube-api-access-c8mj4\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.341978 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.341892    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-tmp-dir\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.342180 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.342156    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-home\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.342249 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.342218    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-model-cache\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.342290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.342274    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-tmp-dir\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.344172 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.344147    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-dshm\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.344424 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.344406    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a215a0c4-42fb-4b21-b6ce-c610120fff14-tls-certs\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.350695 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.350673    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8mj4\" (UniqueName: \"kubernetes.io/projected/a215a0c4-42fb-4b21-b6ce-c610120fff14-kube-api-access-c8mj4\") pod \"rate-test-model-2fae12-kserve-588d79d87-9x42p\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") " pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.392117 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.392062    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:11.665640 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:11.665605    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"]
Sep 11 17:12:11.668837 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:12:11.668809    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda215a0c4_42fb_4b21_b6ce_c610120fff14.slice/crio-16b617a25ec1d69a7488b6e2f030335f4025e15b4b99c7eb48c08196e5a8889b WatchSource:0}: Error finding container 16b617a25ec1d69a7488b6e2f030335f4025e15b4b99c7eb48c08196e5a8889b: Status 404 returned error can't find the container with id 16b617a25ec1d69a7488b6e2f030335f4025e15b4b99c7eb48c08196e5a8889b
Sep 11 17:12:12.252270 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:12.252226    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p" event={"ID":"a215a0c4-42fb-4b21-b6ce-c610120fff14","Type":"ContainerStarted","Data":"e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5"}
Sep 11 17:12:12.252760 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:12.252281    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p" event={"ID":"a215a0c4-42fb-4b21-b6ce-c610120fff14","Type":"ContainerStarted","Data":"16b617a25ec1d69a7488b6e2f030335f4025e15b4b99c7eb48c08196e5a8889b"}
Sep 11 17:12:12.269689 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:12.269633    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p" podStartSLOduration=1.269615918 podStartE2EDuration="1.269615918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:12:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:12:12.26801753 +0000 UTC m=+1728.675236196" watchObservedRunningTime="2026-09-11 17:12:12.269615918 +0000 UTC m=+1728.676834565"
Sep 11 17:12:15.683443 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:15.683408    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:12:17.979106 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:17.979047    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x" podUID="48688c98-f49e-48c2-a01c-924641d320a2" containerName="main" containerID="cri-o://576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa" gracePeriod=45
Sep 11 17:12:18.235413 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.235347    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:12:18.285699 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.285664    2569 generic.go:356] "Generic (PLEG): container finished" podID="48688c98-f49e-48c2-a01c-924641d320a2" containerID="576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa" exitCode=0
Sep 11 17:12:18.285864 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.285704    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"
Sep 11 17:12:18.285864 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.285705    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x" event={"ID":"48688c98-f49e-48c2-a01c-924641d320a2","Type":"ContainerDied","Data":"576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa"}
Sep 11 17:12:18.285864 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.285825    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x" event={"ID":"48688c98-f49e-48c2-a01c-924641d320a2","Type":"ContainerDied","Data":"80021c8c2c6cfdd32a18b4de66cdccb6f24a3932f0edc2a4e55f25ffcd0cf5b1"}
Sep 11 17:12:18.285864 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.285853    2569 scope.go:117] "RemoveContainer" containerID="576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa"
Sep 11 17:12:18.294388 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.294370    2569 scope.go:117] "RemoveContainer" containerID="576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa"
Sep 11 17:12:18.294621 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:18.294596    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa\": container with ID starting with 576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa not found: ID does not exist" containerID="576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa"
Sep 11 17:12:18.294696 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.294628    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa"} err="failed to get container status \"576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa\": rpc error: code = NotFound desc = could not find container \"576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa\": container with ID starting with 576b1c07b222265056c587ce6b92b07cf19e95206044d7bf2efd6aaa0108b3fa not found: ID does not exist"
Sep 11 17:12:18.308983 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.308957    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48688c98-f49e-48c2-a01c-924641d320a2-tls-certs\") pod \"48688c98-f49e-48c2-a01c-924641d320a2\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") "
Sep 11 17:12:18.309095 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.309012    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-model-cache\") pod \"48688c98-f49e-48c2-a01c-924641d320a2\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") "
Sep 11 17:12:18.309095 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.309041    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-dshm\") pod \"48688c98-f49e-48c2-a01c-924641d320a2\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") "
Sep 11 17:12:18.309218 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.309142    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-home\") pod \"48688c98-f49e-48c2-a01c-924641d320a2\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") "
Sep 11 17:12:18.309218 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.309187    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-tmp-dir\") pod \"48688c98-f49e-48c2-a01c-924641d320a2\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") "
Sep 11 17:12:18.309218 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.309214    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm4lh\" (UniqueName: \"kubernetes.io/projected/48688c98-f49e-48c2-a01c-924641d320a2-kube-api-access-wm4lh\") pod \"48688c98-f49e-48c2-a01c-924641d320a2\" (UID: \"48688c98-f49e-48c2-a01c-924641d320a2\") "
Sep 11 17:12:18.309373 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.309283    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-model-cache" (OuterVolumeSpecName: "model-cache") pod "48688c98-f49e-48c2-a01c-924641d320a2" (UID: "48688c98-f49e-48c2-a01c-924641d320a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:12:18.309476 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.309451    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-home" (OuterVolumeSpecName: "home") pod "48688c98-f49e-48c2-a01c-924641d320a2" (UID: "48688c98-f49e-48c2-a01c-924641d320a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:12:18.309541 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.309475    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "48688c98-f49e-48c2-a01c-924641d320a2" (UID: "48688c98-f49e-48c2-a01c-924641d320a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:12:18.309596 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.309539    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:18.309596 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.309554    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:18.311260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.311233    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48688c98-f49e-48c2-a01c-924641d320a2-kube-api-access-wm4lh" (OuterVolumeSpecName: "kube-api-access-wm4lh") pod "48688c98-f49e-48c2-a01c-924641d320a2" (UID: "48688c98-f49e-48c2-a01c-924641d320a2"). InnerVolumeSpecName "kube-api-access-wm4lh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:12:18.311340 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.311260    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48688c98-f49e-48c2-a01c-924641d320a2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "48688c98-f49e-48c2-a01c-924641d320a2" (UID: "48688c98-f49e-48c2-a01c-924641d320a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:12:18.311340 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.311258    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-dshm" (OuterVolumeSpecName: "dshm") pod "48688c98-f49e-48c2-a01c-924641d320a2" (UID: "48688c98-f49e-48c2-a01c-924641d320a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:12:18.410069 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.410033    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:18.410069 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.410062    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm4lh\" (UniqueName: \"kubernetes.io/projected/48688c98-f49e-48c2-a01c-924641d320a2-kube-api-access-wm4lh\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:18.410069 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.410094    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48688c98-f49e-48c2-a01c-924641d320a2-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:18.410069 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.410103    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48688c98-f49e-48c2-a01c-924641d320a2-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:18.607588 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.607554    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"]
Sep 11 17:12:18.609923 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:18.609899    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/sub-test-model-2fae12-kserve-65cdb486f-9xx7x"]
Sep 11 17:12:19.013097 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.013019    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75" podUID="72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" containerName="main" containerID="cri-o://46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4" gracePeriod=45
Sep 11 17:12:19.269573 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.269513    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:12:19.293660 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.293633    2569 generic.go:356] "Generic (PLEG): container finished" podID="72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" containerID="46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4" exitCode=0
Sep 11 17:12:19.293819 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.293681    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"
Sep 11 17:12:19.293819 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.293720    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75" event={"ID":"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8","Type":"ContainerDied","Data":"46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4"}
Sep 11 17:12:19.293819 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.293757    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75" event={"ID":"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8","Type":"ContainerDied","Data":"ac336b8e4afcdf25d07ac4d455b0a30ed234cb66a2c024023d6d0f00e2346f3d"}
Sep 11 17:12:19.293819 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.293778    2569 scope.go:117] "RemoveContainer" containerID="46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4"
Sep 11 17:12:19.303345 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.303328    2569 scope.go:117] "RemoveContainer" containerID="46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4"
Sep 11 17:12:19.303611 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:12:19.303594    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4\": container with ID starting with 46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4 not found: ID does not exist" containerID="46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4"
Sep 11 17:12:19.303665 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.303618    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4"} err="failed to get container status \"46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4\": rpc error: code = NotFound desc = could not find container \"46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4\": container with ID starting with 46b661f0042ea23b4a6eab059956986c5ffa28a99444a00023a737b93d9db7e4 not found: ID does not exist"
Sep 11 17:12:19.318930 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.318907    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvhr2\" (UniqueName: \"kubernetes.io/projected/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-kube-api-access-zvhr2\") pod \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") "
Sep 11 17:12:19.319035 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.318959    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-home\") pod \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") "
Sep 11 17:12:19.319099 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.319034    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tmp-dir\") pod \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") "
Sep 11 17:12:19.319099 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.319053    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tls-certs\") pod \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") "
Sep 11 17:12:19.319207 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.319112    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-dshm\") pod \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") "
Sep 11 17:12:19.319207 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.319160    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-model-cache\") pod \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\" (UID: \"72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8\") "
Sep 11 17:12:19.319317 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.319275    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-home" (OuterVolumeSpecName: "home") pod "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" (UID: "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:12:19.319317 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.319289    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" (UID: "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:12:19.319458 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.319432    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-model-cache" (OuterVolumeSpecName: "model-cache") pod "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" (UID: "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:12:19.319458 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.319450    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:19.319567 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.319466    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:19.321171 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.321147    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-kube-api-access-zvhr2" (OuterVolumeSpecName: "kube-api-access-zvhr2") pod "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" (UID: "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8"). InnerVolumeSpecName "kube-api-access-zvhr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:12:19.321285 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.321262    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" (UID: "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:12:19.321327 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.321305    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-dshm" (OuterVolumeSpecName: "dshm") pod "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" (UID: "72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:12:19.420240 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.420200    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:19.420240 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.420232    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:19.420240 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.420241    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:19.420240 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.420249    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvhr2\" (UniqueName: \"kubernetes.io/projected/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8-kube-api-access-zvhr2\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:12:19.615530 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.615497    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"]
Sep 11 17:12:19.619285 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:19.619258    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/sub-test-model-327a3a-kserve-5dc88c947c-btf75"]
Sep 11 17:12:20.205569 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:20.205526    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48688c98-f49e-48c2-a01c-924641d320a2" path="/var/lib/kubelet/pods/48688c98-f49e-48c2-a01c-924641d320a2/volumes"
Sep 11 17:12:20.206142 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:20.206123    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" path="/var/lib/kubelet/pods/72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8/volumes"
Sep 11 17:12:21.392751 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:21.392721    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:21.393143 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:21.392773    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:21.405201 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:21.405177    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:22.327195 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:22.327163    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:12:29.416436 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:29.416402    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:12:29.517121 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:29.517025    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:12:39.284021 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.283990    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"]
Sep 11 17:12:39.284455 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.284433    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48688c98-f49e-48c2-a01c-924641d320a2" containerName="main"
Sep 11 17:12:39.284499 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.284460    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="48688c98-f49e-48c2-a01c-924641d320a2" containerName="main"
Sep 11 17:12:39.284499 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.284483    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" containerName="main"
Sep 11 17:12:39.284499 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.284492    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" containerName="main"
Sep 11 17:12:39.284630 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.284618    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="72a35d04-cbcf-4cf2-a8da-cbe3735bbdc8" containerName="main"
Sep 11 17:12:39.284682 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.284636    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="48688c98-f49e-48c2-a01c-924641d320a2" containerName="main"
Sep 11 17:12:39.287438 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.287420    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.289471 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.289449    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-327a3a\"/\"kube-root-ca.crt\""
Sep 11 17:12:39.289576 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.289559    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-327a3a\"/\"openshift-service-ca.crt\""
Sep 11 17:12:39.290144 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.290123    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-327a3a\"/\"default-dockercfg-5rzw7\""
Sep 11 17:12:39.290253 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.290146    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-327a3a\"/\"rate-test-model-327a3a-kserve-self-signed-certs\""
Sep 11 17:12:39.299017 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.298994    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"]
Sep 11 17:12:39.322037 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.321749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:12:39.423525 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.423493    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfhbf\" (UniqueName: \"kubernetes.io/projected/e4ed1815-b7d2-4e76-9416-32ae06497824-kube-api-access-pfhbf\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.423729 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.423540    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-tmp-dir\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.423729 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.423561    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-model-cache\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.423729 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.423583    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4ed1815-b7d2-4e76-9416-32ae06497824-tls-certs\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.423729 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.423610    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-home\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.423729 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.423718    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-dshm\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.524995 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.524956    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-home\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.525225 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.525029    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-dshm\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.525225 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.525106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfhbf\" (UniqueName: \"kubernetes.io/projected/e4ed1815-b7d2-4e76-9416-32ae06497824-kube-api-access-pfhbf\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.525225 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.525136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-tmp-dir\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.525225 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.525155    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-model-cache\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.525225 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.525172    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4ed1815-b7d2-4e76-9416-32ae06497824-tls-certs\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.525578 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.525534    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-home\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.525578 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.525562    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-model-cache\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.525770 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.525631    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-tmp-dir\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.527442 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.527422    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-dshm\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.527692 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.527675    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4ed1815-b7d2-4e76-9416-32ae06497824-tls-certs\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.534197 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.534130    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfhbf\" (UniqueName: \"kubernetes.io/projected/e4ed1815-b7d2-4e76-9416-32ae06497824-kube-api-access-pfhbf\") pod \"rate-test-model-327a3a-kserve-6d77989f79-jtf6l\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") " pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.597028 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.596988    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:39.872507 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:39.870320    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"]
Sep 11 17:12:40.406698 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:40.406655    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l" event={"ID":"e4ed1815-b7d2-4e76-9416-32ae06497824","Type":"ContainerStarted","Data":"d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48"}
Sep 11 17:12:40.406698 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:40.406704    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l" event={"ID":"e4ed1815-b7d2-4e76-9416-32ae06497824","Type":"ContainerStarted","Data":"a05c8120964b4b474c434074baf6babbc4fd99dd31af74f871f9b77d1beeca67"}
Sep 11 17:12:40.424355 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:40.424297    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l" podStartSLOduration=1.424278695 podStartE2EDuration="1.424278695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:12:40.422746793 +0000 UTC m=+1756.829965452" watchObservedRunningTime="2026-09-11 17:12:40.424278695 +0000 UTC m=+1756.831497343"
Sep 11 17:12:43.081980 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:43.081945    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:12:49.597173 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:49.597119    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:49.597577 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:49.597196    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:49.609848 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:49.609814    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:12:50.469317 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:12:50.469283    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:13:09.389946 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:09.389901    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:11.988980 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:11.988938    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:13:19.335488 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:19.335402    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:22.381279 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:22.381245    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:13:27.975392 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:27.975353    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"]
Sep 11 17:13:27.995329 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:27.995255    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:28.971096 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:28.971039    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"]
Sep 11 17:13:29.361181 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.361060    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:29.426550 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.426521    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:29.675790 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.675760    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"]
Sep 11 17:13:29.679522 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.679499    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.681357 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.681332    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-269ba8-openshift-default-dockercfg-hp62j\""
Sep 11 17:13:29.689564 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.689539    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"]
Sep 11 17:13:29.705463 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.705432    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:29.722050 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.722017    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-data\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.722050 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.722052    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-certs\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.722252 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.722111    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b039755-032a-4ee7-889b-c3bbddf80155-istiod-ca-cert\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.722252 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.722136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-envoy\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.722252 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.722183    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-istio-token\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.722252 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.722222    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b039755-032a-4ee7-889b-c3bbddf80155-istio-podinfo\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.722384 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.722268    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-socket\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.722384 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.722319    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-credential-socket\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.722384 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.722358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb2jc\" (UniqueName: \"kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-kube-api-access-lb2jc\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.804937 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.804906    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:29.823716 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.823689    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-credential-socket\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.823855 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.823726    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb2jc\" (UniqueName: \"kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-kube-api-access-lb2jc\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.823855 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.823755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-data\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.823855 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.823777    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-certs\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.823855 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.823807    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b039755-032a-4ee7-889b-c3bbddf80155-istiod-ca-cert\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.823855 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.823826    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-envoy\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.824157 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.823871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-istio-token\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.824157 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.824012    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b039755-032a-4ee7-889b-c3bbddf80155-istio-podinfo\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.824157 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.824100    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-socket\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.824157 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.824147    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-credential-socket\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.824325 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.824195    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-data\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.824325 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.824276    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-certs\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.824408 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.824390    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-socket\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.824583 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.824560    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b039755-032a-4ee7-889b-c3bbddf80155-istiod-ca-cert\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.826476 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.826455    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-envoy\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.826595 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.826579    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b039755-032a-4ee7-889b-c3bbddf80155-istio-podinfo\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.832609 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.832585    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-istio-token\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.832742 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.832633    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb2jc\" (UniqueName: \"kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-kube-api-access-lb2jc\") pod \"e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") " pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:29.991649 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:29.991561    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:30.275688 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:30.275656    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"]
Sep 11 17:13:30.277304 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:13:30.277273    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b039755_032a_4ee7_889b_c3bbddf80155.slice/crio-11f25986f1e5fc1e5cb5728e663ead1d03e930d44d46444502eb78cb17cdf642 WatchSource:0}: Error finding container 11f25986f1e5fc1e5cb5728e663ead1d03e930d44d46444502eb78cb17cdf642: Status 404 returned error can't find the container with id 11f25986f1e5fc1e5cb5728e663ead1d03e930d44d46444502eb78cb17cdf642
Sep 11 17:13:30.279561 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:30.279533    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:13:30.279654 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:30.279597    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:13:30.279654 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:30.279622    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:13:30.666681 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:30.666643    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" event={"ID":"3b039755-032a-4ee7-889b-c3bbddf80155","Type":"ContainerStarted","Data":"1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4"}
Sep 11 17:13:30.667096 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:30.666704    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" event={"ID":"3b039755-032a-4ee7-889b-c3bbddf80155","Type":"ContainerStarted","Data":"11f25986f1e5fc1e5cb5728e663ead1d03e930d44d46444502eb78cb17cdf642"}
Sep 11 17:13:30.688310 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:30.688258    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" podStartSLOduration=1.688240721 podStartE2EDuration="1.688240721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:13:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:13:30.686915358 +0000 UTC m=+1807.094134029" watchObservedRunningTime="2026-09-11 17:13:30.688240721 +0000 UTC m=+1807.095459368"
Sep 11 17:13:30.991963 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:30.991875    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:31.997509 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:31.997478    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:32.051023 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:32.050990    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:32.578324 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:32.578269    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:32.612005 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:32.611967    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:13:32.668608 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:32.668573    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:32.677712 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:32.677686    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:32.678732 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:32.678714    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:32.929656 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:32.929616    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:33.690565 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:33.690530    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:34.512784 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:34.512742    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:34.596344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:34.596235    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:35.037815 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:35.037781    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:35.138409 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:35.138380    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:35.260246 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:35.260215    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:36.026149 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:36.026114    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:41.887160 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:41.887126    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:13:42.994675 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:42.994626    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p" podUID="a215a0c4-42fb-4b21-b6ce-c610120fff14" containerName="main" containerID="cri-o://e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5" gracePeriod=45
Sep 11 17:13:43.283280 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.283252    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:13:43.374378 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374341    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-dshm\") pod \"a215a0c4-42fb-4b21-b6ce-c610120fff14\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") "
Sep 11 17:13:43.374562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374394    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-tmp-dir\") pod \"a215a0c4-42fb-4b21-b6ce-c610120fff14\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") "
Sep 11 17:13:43.374562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374492    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-model-cache\") pod \"a215a0c4-42fb-4b21-b6ce-c610120fff14\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") "
Sep 11 17:13:43.374562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374511    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-home\") pod \"a215a0c4-42fb-4b21-b6ce-c610120fff14\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") "
Sep 11 17:13:43.374562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374539    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8mj4\" (UniqueName: \"kubernetes.io/projected/a215a0c4-42fb-4b21-b6ce-c610120fff14-kube-api-access-c8mj4\") pod \"a215a0c4-42fb-4b21-b6ce-c610120fff14\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") "
Sep 11 17:13:43.374787 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374586    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a215a0c4-42fb-4b21-b6ce-c610120fff14-tls-certs\") pod \"a215a0c4-42fb-4b21-b6ce-c610120fff14\" (UID: \"a215a0c4-42fb-4b21-b6ce-c610120fff14\") "
Sep 11 17:13:43.374787 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374652    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a215a0c4-42fb-4b21-b6ce-c610120fff14" (UID: "a215a0c4-42fb-4b21-b6ce-c610120fff14"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:13:43.374787 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374758    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-home" (OuterVolumeSpecName: "home") pod "a215a0c4-42fb-4b21-b6ce-c610120fff14" (UID: "a215a0c4-42fb-4b21-b6ce-c610120fff14"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:13:43.374787 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374769    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-model-cache" (OuterVolumeSpecName: "model-cache") pod "a215a0c4-42fb-4b21-b6ce-c610120fff14" (UID: "a215a0c4-42fb-4b21-b6ce-c610120fff14"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:13:43.374986 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374964    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:43.375035 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374984    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:43.375035 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.374999    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:43.376699 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.376678    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a215a0c4-42fb-4b21-b6ce-c610120fff14-kube-api-access-c8mj4" (OuterVolumeSpecName: "kube-api-access-c8mj4") pod "a215a0c4-42fb-4b21-b6ce-c610120fff14" (UID: "a215a0c4-42fb-4b21-b6ce-c610120fff14"). InnerVolumeSpecName "kube-api-access-c8mj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:13:43.376771 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.376678    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a215a0c4-42fb-4b21-b6ce-c610120fff14-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a215a0c4-42fb-4b21-b6ce-c610120fff14" (UID: "a215a0c4-42fb-4b21-b6ce-c610120fff14"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:13:43.376771 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.376746    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-dshm" (OuterVolumeSpecName: "dshm") pod "a215a0c4-42fb-4b21-b6ce-c610120fff14" (UID: "a215a0c4-42fb-4b21-b6ce-c610120fff14"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:13:43.476020 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.475979    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8mj4\" (UniqueName: \"kubernetes.io/projected/a215a0c4-42fb-4b21-b6ce-c610120fff14-kube-api-access-c8mj4\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:43.476020 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.476014    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a215a0c4-42fb-4b21-b6ce-c610120fff14-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:43.476020 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.476025    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a215a0c4-42fb-4b21-b6ce-c610120fff14-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:43.738452 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.738409    2569 generic.go:356] "Generic (PLEG): container finished" podID="a215a0c4-42fb-4b21-b6ce-c610120fff14" containerID="e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5" exitCode=0
Sep 11 17:13:43.738660 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.738585    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p" event={"ID":"a215a0c4-42fb-4b21-b6ce-c610120fff14","Type":"ContainerDied","Data":"e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5"}
Sep 11 17:13:43.738660 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.738624    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p" event={"ID":"a215a0c4-42fb-4b21-b6ce-c610120fff14","Type":"ContainerDied","Data":"16b617a25ec1d69a7488b6e2f030335f4025e15b4b99c7eb48c08196e5a8889b"}
Sep 11 17:13:43.738660 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.738646    2569 scope.go:117] "RemoveContainer" containerID="e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5"
Sep 11 17:13:43.738836 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.738825    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"
Sep 11 17:13:43.760465 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.760441    2569 scope.go:117] "RemoveContainer" containerID="e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5"
Sep 11 17:13:43.760846 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:13:43.760822    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5\": container with ID starting with e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5 not found: ID does not exist" containerID="e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5"
Sep 11 17:13:43.760924 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.760855    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5"} err="failed to get container status \"e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5\": rpc error: code = NotFound desc = could not find container \"e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5\": container with ID starting with e26142cf764b1452bc4fccab6a5bd5dcb80911cca25c19b48db17e6c00e144a5 not found: ID does not exist"
Sep 11 17:13:43.767746 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.767716    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"]
Sep 11 17:13:43.771654 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.771628    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fae12/rate-test-model-2fae12-kserve-588d79d87-9x42p"]
Sep 11 17:13:43.995447 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:43.995355    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l" podUID="e4ed1815-b7d2-4e76-9416-32ae06497824" containerName="main" containerID="cri-o://d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48" gracePeriod=45
Sep 11 17:13:44.208499 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.208463    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a215a0c4-42fb-4b21-b6ce-c610120fff14" path="/var/lib/kubelet/pods/a215a0c4-42fb-4b21-b6ce-c610120fff14/volumes"
Sep 11 17:13:44.284956 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.284931    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:13:44.385328 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.385287    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-dshm\") pod \"e4ed1815-b7d2-4e76-9416-32ae06497824\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") "
Sep 11 17:13:44.385540 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.385372    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4ed1815-b7d2-4e76-9416-32ae06497824-tls-certs\") pod \"e4ed1815-b7d2-4e76-9416-32ae06497824\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") "
Sep 11 17:13:44.385540 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.385446    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-home\") pod \"e4ed1815-b7d2-4e76-9416-32ae06497824\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") "
Sep 11 17:13:44.385540 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.385532    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-tmp-dir\") pod \"e4ed1815-b7d2-4e76-9416-32ae06497824\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") "
Sep 11 17:13:44.385727 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.385588    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfhbf\" (UniqueName: \"kubernetes.io/projected/e4ed1815-b7d2-4e76-9416-32ae06497824-kube-api-access-pfhbf\") pod \"e4ed1815-b7d2-4e76-9416-32ae06497824\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") "
Sep 11 17:13:44.385727 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.385630    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-model-cache\") pod \"e4ed1815-b7d2-4e76-9416-32ae06497824\" (UID: \"e4ed1815-b7d2-4e76-9416-32ae06497824\") "
Sep 11 17:13:44.385727 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.385677    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-home" (OuterVolumeSpecName: "home") pod "e4ed1815-b7d2-4e76-9416-32ae06497824" (UID: "e4ed1815-b7d2-4e76-9416-32ae06497824"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:13:44.385903 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.385728    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e4ed1815-b7d2-4e76-9416-32ae06497824" (UID: "e4ed1815-b7d2-4e76-9416-32ae06497824"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:13:44.385903 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.385846    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-model-cache" (OuterVolumeSpecName: "model-cache") pod "e4ed1815-b7d2-4e76-9416-32ae06497824" (UID: "e4ed1815-b7d2-4e76-9416-32ae06497824"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:13:44.386019 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.385997    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:44.386117 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.386024    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:44.386117 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.386038    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:44.387660 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.387618    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-dshm" (OuterVolumeSpecName: "dshm") pod "e4ed1815-b7d2-4e76-9416-32ae06497824" (UID: "e4ed1815-b7d2-4e76-9416-32ae06497824"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:13:44.387660 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.387636    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ed1815-b7d2-4e76-9416-32ae06497824-kube-api-access-pfhbf" (OuterVolumeSpecName: "kube-api-access-pfhbf") pod "e4ed1815-b7d2-4e76-9416-32ae06497824" (UID: "e4ed1815-b7d2-4e76-9416-32ae06497824"). InnerVolumeSpecName "kube-api-access-pfhbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:13:44.387780 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.387710    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4ed1815-b7d2-4e76-9416-32ae06497824-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e4ed1815-b7d2-4e76-9416-32ae06497824" (UID: "e4ed1815-b7d2-4e76-9416-32ae06497824"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:13:44.487547 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.487500    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfhbf\" (UniqueName: \"kubernetes.io/projected/e4ed1815-b7d2-4e76-9416-32ae06497824-kube-api-access-pfhbf\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:44.487547 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.487543    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4ed1815-b7d2-4e76-9416-32ae06497824-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:44.487547 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.487556    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4ed1815-b7d2-4e76-9416-32ae06497824-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:13:44.748224 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.748185    2569 generic.go:356] "Generic (PLEG): container finished" podID="e4ed1815-b7d2-4e76-9416-32ae06497824" containerID="d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48" exitCode=0
Sep 11 17:13:44.748441 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.748237    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"
Sep 11 17:13:44.748441 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.748274    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l" event={"ID":"e4ed1815-b7d2-4e76-9416-32ae06497824","Type":"ContainerDied","Data":"d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48"}
Sep 11 17:13:44.748441 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.748314    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l" event={"ID":"e4ed1815-b7d2-4e76-9416-32ae06497824","Type":"ContainerDied","Data":"a05c8120964b4b474c434074baf6babbc4fd99dd31af74f871f9b77d1beeca67"}
Sep 11 17:13:44.748441 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.748337    2569 scope.go:117] "RemoveContainer" containerID="d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48"
Sep 11 17:13:44.758995 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.758973    2569 scope.go:117] "RemoveContainer" containerID="d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48"
Sep 11 17:13:44.759341 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:13:44.759320    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48\": container with ID starting with d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48 not found: ID does not exist" containerID="d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48"
Sep 11 17:13:44.759412 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.759347    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48"} err="failed to get container status \"d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48\": rpc error: code = NotFound desc = could not find container \"d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48\": container with ID starting with d72361e88b8ae6b2e2c3ecbd806a5b3b03366466f13acc9e6d1a3dabe8b4aa48 not found: ID does not exist"
Sep 11 17:13:44.769150 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.769114    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"]
Sep 11 17:13:44.774955 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:44.774923    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-327a3a/rate-test-model-327a3a-kserve-6d77989f79-jtf6l"]
Sep 11 17:13:46.206894 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:46.206855    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4ed1815-b7d2-4e76-9416-32ae06497824" path="/var/lib/kubelet/pods/e4ed1815-b7d2-4e76-9416-32ae06497824/volumes"
Sep 11 17:13:47.742410 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:47.742143    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": read tcp 10.134.0.2:41434->10.134.0.84:15021: read: connection reset by peer" start-of-body=
Sep 11 17:13:47.742410 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:47.742208    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": read tcp 10.134.0.2:41434->10.134.0.84:15021: read: connection reset by peer"
Sep 11 17:13:47.770020 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:47.769988    2569 generic.go:356] "Generic (PLEG): container finished" podID="3b039755-032a-4ee7-889b-c3bbddf80155" containerID="1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4" exitCode=0
Sep 11 17:13:47.770180 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:47.770060    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" event={"ID":"3b039755-032a-4ee7-889b-c3bbddf80155","Type":"ContainerDied","Data":"1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4"}
Sep 11 17:13:47.770553 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:47.770535    2569 scope.go:117] "RemoveContainer" containerID="1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4"
Sep 11 17:13:47.771746 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:47.771715    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:13:47.771820 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:47.771801    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:13:47.771860 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:47.771845    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:13:48.777447 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:48.777411    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" event={"ID":"3b039755-032a-4ee7-889b-c3bbddf80155","Type":"ContainerStarted","Data":"a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a"}
Sep 11 17:13:48.991948 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:48.991907    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:48.992145 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:48.992116    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 11 17:13:48.992199 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:48.992182    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 11 17:13:49.992694 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:49.992654    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 11 17:13:49.993137 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:49.992713    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 11 17:13:50.992275 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:50.992232    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 11 17:13:50.992484 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:50.992295    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 11 17:13:51.992888 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:51.992851    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 11 17:13:51.993335 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:51.992913    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 11 17:13:52.992889 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:52.992850    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 11 17:13:52.993101 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:52.992902    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 11 17:13:53.992017 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:53.991976    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 11 17:13:53.992224 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:53.992061    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 11 17:13:54.995783 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:54.995753    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:54.996213 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:54.996038    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:54.996849 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:54.996832    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:13:56.852626 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:56.852563    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:57.511246 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:57.511203    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:57.653782 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:57.653733    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:58.170841 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.170761    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:58.373761 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.373710    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:58.466966 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.466886    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"]
Sep 11 17:13:58.467330 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.467316    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ed1815-b7d2-4e76-9416-32ae06497824" containerName="main"
Sep 11 17:13:58.467385 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.467331    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ed1815-b7d2-4e76-9416-32ae06497824" containerName="main"
Sep 11 17:13:58.467385 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.467356    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a215a0c4-42fb-4b21-b6ce-c610120fff14" containerName="main"
Sep 11 17:13:58.467385 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.467362    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a215a0c4-42fb-4b21-b6ce-c610120fff14" containerName="main"
Sep 11 17:13:58.467480 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.467423    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a215a0c4-42fb-4b21-b6ce-c610120fff14" containerName="main"
Sep 11 17:13:58.467480 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.467432    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ed1815-b7d2-4e76-9416-32ae06497824" containerName="main"
Sep 11 17:13:58.469429 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.469410    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.474513 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.474490    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-750358-openshift-default-dockercfg-qzkpv\""
Sep 11 17:13:58.487838 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.487800    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"]
Sep 11 17:13:58.498739 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.498676    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:58.526733 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.526692    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-socket\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.526908 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.526755    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-data\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.526908 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.526782    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-envoy\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.526908 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.526836    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/038a0922-55f7-40c9-8445-722bb7702728-istiod-ca-cert\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.527026 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.526909    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/038a0922-55f7-40c9-8445-722bb7702728-istio-podinfo\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.527026 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.526952    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-credential-socket\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.527026 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.527006    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpq8j\" (UniqueName: \"kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-kube-api-access-hpq8j\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.527168 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.527065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-certs\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.527205 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.527171    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-istio-token\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.627755 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.627719    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/038a0922-55f7-40c9-8445-722bb7702728-istio-podinfo\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.627948 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.627765    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-credential-socket\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.627948 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.627795    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpq8j\" (UniqueName: \"kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-kube-api-access-hpq8j\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.627948 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.627814    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-certs\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.627948 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.627855    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-istio-token\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.627948 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.627932    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-socket\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.628232 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.627961    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-data\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.628232 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.627987    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-envoy\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.628232 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.628017    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/038a0922-55f7-40c9-8445-722bb7702728-istiod-ca-cert\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.628394 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.628321    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-credential-socket\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.628394 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.628366    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-certs\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.628499 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.628398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-socket\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.628499 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.628481    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-data\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.628807 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.628787    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/038a0922-55f7-40c9-8445-722bb7702728-istiod-ca-cert\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.630299 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.630279    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-envoy\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.630408 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.630362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/038a0922-55f7-40c9-8445-722bb7702728-istio-podinfo\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.635153 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.635130    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-istio-token\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.635369 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.635351    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpq8j\" (UniqueName: \"kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-kube-api-access-hpq8j\") pod \"e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") " pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:58.780518 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:58.780414    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:13:59.057562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:59.057478    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"]
Sep 11 17:13:59.060836 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:13:59.060809    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod038a0922_55f7_40c9_8445_722bb7702728.slice/crio-80aa75c94d5df910bd6d00f5c67e52fea57861bcb31b148857f04dc254c905c8 WatchSource:0}: Error finding container 80aa75c94d5df910bd6d00f5c67e52fea57861bcb31b148857f04dc254c905c8: Status 404 returned error can't find the container with id 80aa75c94d5df910bd6d00f5c67e52fea57861bcb31b148857f04dc254c905c8
Sep 11 17:13:59.063512 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:59.063481    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:13:59.063606 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:59.063546    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:13:59.063606 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:59.063576    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:13:59.089489 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:59.089452    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:59.196704 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:59.196666    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:13:59.845704 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:59.845662    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" event={"ID":"038a0922-55f7-40c9-8445-722bb7702728","Type":"ContainerStarted","Data":"1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393"}
Sep 11 17:13:59.845883 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:59.845726    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" event={"ID":"038a0922-55f7-40c9-8445-722bb7702728","Type":"ContainerStarted","Data":"80aa75c94d5df910bd6d00f5c67e52fea57861bcb31b148857f04dc254c905c8"}
Sep 11 17:13:59.871420 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:13:59.871362    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" podStartSLOduration=1.871346005 podStartE2EDuration="1.871346005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:13:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:13:59.8697256 +0000 UTC m=+1836.276944248" watchObservedRunningTime="2026-09-11 17:13:59.871346005 +0000 UTC m=+1836.278564655"
Sep 11 17:14:00.780935 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:00.780896    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:14:00.785721 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:00.785693    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:14:00.850888 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:00.850853    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:14:00.851891 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:00.851869    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:14:01.082424 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:01.082343    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:01.774885 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:01.774851    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:01.872871 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:01.872824    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:02.682230 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:02.682192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:03.486775 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:03.486744    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:04.484937 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.484902    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"]
Sep 11 17:14:04.488666 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.488642    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.490819 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.490794    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-750358-serving-cert\""
Sep 11 17:14:04.491235 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.491215    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 17:14:04.491524 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.491505    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 17:14:04.491524 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.491522    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8lxr4\""
Sep 11 17:14:04.491679 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.491559    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:14:04.502543 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.502512    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"]
Sep 11 17:14:04.538858 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.538819    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:04.593127 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.591360    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.593127 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.591451    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kjbz\" (UniqueName: \"kubernetes.io/projected/167913a1-309f-432c-a3be-5b7b3544c52a-kube-api-access-9kjbz\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.593127 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.591553    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/167913a1-309f-432c-a3be-5b7b3544c52a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.593127 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.591594    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-metrics-certs\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.655167 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.655109    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:04.692302 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.692267    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/167913a1-309f-432c-a3be-5b7b3544c52a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.692476 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.692307    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-metrics-certs\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.692476 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.692380    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.692476 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.692431    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kjbz\" (UniqueName: \"kubernetes.io/projected/167913a1-309f-432c-a3be-5b7b3544c52a-kube-api-access-9kjbz\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.693022 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.692999    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/167913a1-309f-432c-a3be-5b7b3544c52a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.694925 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.694904    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.695021 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.694933    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-metrics-certs\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.701015 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.700992    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kjbz\" (UniqueName: \"kubernetes.io/projected/167913a1-309f-432c-a3be-5b7b3544c52a-kube-api-access-9kjbz\") pod \"maas-api-e2e-ipp-b-750358-65ff49769f-pm569\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:04.739147 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.739034    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:04.801845 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:04.801800    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:05.042819 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:05.042728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:05.081427 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:05.081396    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"]
Sep 11 17:14:05.082407 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:14:05.082377    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod167913a1_309f_432c_a3be_5b7b3544c52a.slice/crio-222834f5f3e166fc95d323ed858382bb1cecc4d8878251bb4be7319a6aba60e8 WatchSource:0}: Error finding container 222834f5f3e166fc95d323ed858382bb1cecc4d8878251bb4be7319a6aba60e8: Status 404 returned error can't find the container with id 222834f5f3e166fc95d323ed858382bb1cecc4d8878251bb4be7319a6aba60e8
Sep 11 17:14:05.143743 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:05.143705    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:05.881387 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:05.881341    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569" event={"ID":"167913a1-309f-432c-a3be-5b7b3544c52a","Type":"ContainerStarted","Data":"222834f5f3e166fc95d323ed858382bb1cecc4d8878251bb4be7319a6aba60e8"}
Sep 11 17:14:07.893504 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:07.893465    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569" event={"ID":"167913a1-309f-432c-a3be-5b7b3544c52a","Type":"ContainerStarted","Data":"7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15"}
Sep 11 17:14:07.893924 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:07.893626    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:07.912439 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:07.912381    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569" podStartSLOduration=1.566324912 podStartE2EDuration="3.912365273s" totalImagesPullingTime="2.346040361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:14:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:14:07.911192146 +0000 UTC m=+1844.318410815" watchObservedRunningTime="2026-09-11 17:14:07.912365273 +0000 UTC m=+1844.319583921"
Sep 11 17:14:09.379433 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:09.379390    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:14:13.904761 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:13.904731    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:14:19.960705 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:19.960666    2569 generic.go:356] "Generic (PLEG): container finished" podID="038a0922-55f7-40c9-8445-722bb7702728" containerID="1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393" exitCode=0
Sep 11 17:14:19.961180 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:19.960741    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" event={"ID":"038a0922-55f7-40c9-8445-722bb7702728","Type":"ContainerDied","Data":"1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393"}
Sep 11 17:14:19.961180 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:19.961172    2569 scope.go:117] "RemoveContainer" containerID="1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393"
Sep 11 17:14:19.962272 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:19.962244    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:14:19.962339 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:19.962314    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:14:19.962375 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:19.962343    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:14:20.967977 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:20.967935    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" event={"ID":"038a0922-55f7-40c9-8445-722bb7702728","Type":"ContainerStarted","Data":"0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b"}
Sep 11 17:14:21.781332 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:21.781296    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:14:21.781564 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:21.781538    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 11 17:14:21.781618 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:21.781591    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 11 17:14:22.781682 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:22.781638    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 11 17:14:22.782130 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:22.781700    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 11 17:14:23.781266 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:23.781225    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 11 17:14:23.781452 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:23.781281    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 11 17:14:24.487578 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:24.487545    2569 scope.go:117] "RemoveContainer" containerID="ce737539870809257796c03e423dbf68447cb755c6dbca3d28267953a63e6fcd"
Sep 11 17:14:24.497090 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:24.497045    2569 scope.go:117] "RemoveContainer" containerID="c531f30df4ec9f02b617643a5b2de21ec9f2fe4efadbfb09217002a27f1f2ef9"
Sep 11 17:14:24.781230 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:24.781136    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 11 17:14:24.781230 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:24.781194    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 11 17:14:25.781775 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:25.781737    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 11 17:14:25.782209 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:25.781795    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 11 17:14:26.550092 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:26.550032    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:26.741277 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:26.741234    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:26.785791 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:26.785755    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:14:26.786260 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:26.786046    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:14:26.786714 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:26.786699    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:14:27.467834 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:27.467799    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:28.227930 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:28.227895    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:28.947971 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:28.947942    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:29.830397 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:29.830327    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:30.542010 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:30.541975    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:34.176990 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.176958    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"]
Sep 11 17:14:34.179763 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.179740    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.182026 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.182001    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-269ba8\"/\"ipp-route-269ba8-kserve-self-signed-certs\""
Sep 11 17:14:34.182182 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.182057    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-269ba8\"/\"openshift-service-ca.crt\""
Sep 11 17:14:34.182182 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.182114    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-269ba8\"/\"kube-root-ca.crt\""
Sep 11 17:14:34.182484 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.182466    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-269ba8\"/\"default-dockercfg-nmxvv\""
Sep 11 17:14:34.190277 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.190242    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"]
Sep 11 17:14:34.209492 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.209440    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:34.291268 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.291219    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-tmp-dir\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.291464 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.291286    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnt9x\" (UniqueName: \"kubernetes.io/projected/e38b8f95-ba39-4a2a-a157-8c790496741c-kube-api-access-gnt9x\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.291464 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.291309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-home\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.291464 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.291341    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-dshm\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.291464 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.291441    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e38b8f95-ba39-4a2a-a157-8c790496741c-tls-certs\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.291628 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.291494    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-model-cache\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.392250 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.392202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-tmp-dir\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.392518 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.392289    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnt9x\" (UniqueName: \"kubernetes.io/projected/e38b8f95-ba39-4a2a-a157-8c790496741c-kube-api-access-gnt9x\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.392518 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.392318    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-home\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.392518 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.392458    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-dshm\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.392518 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.392512    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e38b8f95-ba39-4a2a-a157-8c790496741c-tls-certs\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.392755 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.392555    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-model-cache\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.392755 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.392688    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-tmp-dir\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.392755 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.392710    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-home\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.392912 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.392892    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-model-cache\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.394862 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.394834    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-dshm\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.395039 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.395022    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e38b8f95-ba39-4a2a-a157-8c790496741c-tls-certs\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.401047 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.401017    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnt9x\" (UniqueName: \"kubernetes.io/projected/e38b8f95-ba39-4a2a-a157-8c790496741c-kube-api-access-gnt9x\") pod \"ipp-route-269ba8-kserve-56b89fccf-nmxqh\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") " pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.490958 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.490860    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:34.706337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.706303    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:34.772460 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.772435    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"]
Sep 11 17:14:34.774643 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:14:34.774595    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode38b8f95_ba39_4a2a_a157_8c790496741c.slice/crio-2d1b5ced62cde41a21816eaab307a0bd0899e4f18afce0dc877c597165d734d6 WatchSource:0}: Error finding container 2d1b5ced62cde41a21816eaab307a0bd0899e4f18afce0dc877c597165d734d6: Status 404 returned error can't find the container with id 2d1b5ced62cde41a21816eaab307a0bd0899e4f18afce0dc877c597165d734d6
Sep 11 17:14:34.774778 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:34.774669    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:35.039848 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:35.039751    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh" event={"ID":"e38b8f95-ba39-4a2a-a157-8c790496741c","Type":"ContainerStarted","Data":"e0f13c182f9c6b94b970e44939b0189fd25a4f4a84084f5973e4e458cce6a747"}
Sep 11 17:14:35.039848 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:35.039798    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh" event={"ID":"e38b8f95-ba39-4a2a-a157-8c790496741c","Type":"ContainerStarted","Data":"2d1b5ced62cde41a21816eaab307a0bd0899e4f18afce0dc877c597165d734d6"}
Sep 11 17:14:35.058693 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:35.058632    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh" podStartSLOduration=1.058612509 podStartE2EDuration="1.058612509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:14:35.056477835 +0000 UTC m=+1871.463696519" watchObservedRunningTime="2026-09-11 17:14:35.058612509 +0000 UTC m=+1871.465831157"
Sep 11 17:14:38.388486 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:38.388441    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:14:44.491883 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:44.491833    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:44.492377 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:44.491897    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:44.505866 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:44.505839    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:45.103330 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:45.103298    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:14:51.479453 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:51.479367    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:14:57.881270 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:57.881234    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:58.171275 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.171194    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"]
Sep 11 17:14:58.175219 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.175199    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.177503 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.177480    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-891cc8-openshift-default-dockercfg-26mdw\""
Sep 11 17:14:58.188365 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.188335    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"]
Sep 11 17:14:58.190543 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.190505    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:58.319632 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.319598    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:14:58.339932 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.339903    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-istio-token\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.340098 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.339948    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-envoy\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.340098 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.339978    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjzvb\" (UniqueName: \"kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-kube-api-access-bjzvb\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.340098 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.340035    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/382f4664-7c0c-427d-8263-1aac516dc26e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.340218 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.340136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-certs\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.340218 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.340168    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-credential-socket\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.340218 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.340184    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/382f4664-7c0c-427d-8263-1aac516dc26e-istio-podinfo\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.340318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.340224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-socket\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.340318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.340253    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-data\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.441683 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.441599    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-envoy\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.441683 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.441648    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjzvb\" (UniqueName: \"kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-kube-api-access-bjzvb\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.441683 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.441678    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/382f4664-7c0c-427d-8263-1aac516dc26e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.441938 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.441734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-certs\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.441938 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.441773    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-credential-socket\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.441938 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.441796    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/382f4664-7c0c-427d-8263-1aac516dc26e-istio-podinfo\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.441938 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.441825    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-socket\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.441938 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.441852    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-data\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.441938 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.441928    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-istio-token\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.442277 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.442214    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-certs\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.442277 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.442252    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-socket\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.442387 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.442311    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-data\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.442490 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.442468    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-credential-socket\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.442490 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.442483    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/382f4664-7c0c-427d-8263-1aac516dc26e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.443963 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.443944    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-envoy\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.444298 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.444281    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/382f4664-7c0c-427d-8263-1aac516dc26e-istio-podinfo\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.449249 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.449224    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-istio-token\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.449491 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.449383    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjzvb\" (UniqueName: \"kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-kube-api-access-bjzvb\") pod \"e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") " pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.486450 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.486418    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:58.761537 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.761512    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"]
Sep 11 17:14:58.763002 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:14:58.762972    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod382f4664_7c0c_427d_8263_1aac516dc26e.slice/crio-a471f896b1877019528e92b9390353a5dcb3708bc4ca26fd164562c303efd622 WatchSource:0}: Error finding container a471f896b1877019528e92b9390353a5dcb3708bc4ca26fd164562c303efd622: Status 404 returned error can't find the container with id a471f896b1877019528e92b9390353a5dcb3708bc4ca26fd164562c303efd622
Sep 11 17:14:58.765346 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.765315    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:14:58.765433 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.765378    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:14:58.765433 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:58.765409    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:14:59.168990 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:59.168952    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" event={"ID":"382f4664-7c0c-427d-8263-1aac516dc26e","Type":"ContainerStarted","Data":"f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5"}
Sep 11 17:14:59.169449 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:59.169018    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" event={"ID":"382f4664-7c0c-427d-8263-1aac516dc26e","Type":"ContainerStarted","Data":"a471f896b1877019528e92b9390353a5dcb3708bc4ca26fd164562c303efd622"}
Sep 11 17:14:59.190191 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:59.190137    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" podStartSLOduration=1.190120912 podStartE2EDuration="1.190120912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:14:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:14:59.189145419 +0000 UTC m=+1895.596364079" watchObservedRunningTime="2026-09-11 17:14:59.190120912 +0000 UTC m=+1895.597339557"
Sep 11 17:14:59.486897 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:59.486807    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:14:59.492041 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:14:59.492014    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:15:00.136877 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.136839    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb"]
Sep 11 17:15:00.140782 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.140758    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n"]
Sep 11 17:15:00.140923 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.140872    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb"
Sep 11 17:15:00.143883 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.143858    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 17:15:00.144530 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.144508    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n"
Sep 11 17:15:00.154798 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.154774    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb"
Sep 11 17:15:00.156713 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.156669    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb"]
Sep 11 17:15:00.159425 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.159389    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n"]
Sep 11 17:15:00.164047 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.164026    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n"
Sep 11 17:15:00.186150 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.185941    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:15:00.187313 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.187282    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:15:00.442001 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.441972    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb"]
Sep 11 17:15:00.444001 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:15:00.443976    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09623531_959d_4d48_b153_4accd0a14188.slice/crio-d8778f97cd091b088775722832bba6bf825a3f0d1e5240ef59cdb70332095106 WatchSource:0}: Error finding container d8778f97cd091b088775722832bba6bf825a3f0d1e5240ef59cdb70332095106: Status 404 returned error can't find the container with id d8778f97cd091b088775722832bba6bf825a3f0d1e5240ef59cdb70332095106
Sep 11 17:15:00.449786 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.449762    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:00.454940 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:00.454916    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n"]
Sep 11 17:15:00.459191 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:15:00.459058    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a16751d_0d92_481b_9b41_1392035b51bd.slice/crio-b9a97972d8fa28b8aa94d05c8122a26b2ed58e68b5667a09f66a41381251c573 WatchSource:0}: Error finding container b9a97972d8fa28b8aa94d05c8122a26b2ed58e68b5667a09f66a41381251c573: Status 404 returned error can't find the container with id b9a97972d8fa28b8aa94d05c8122a26b2ed58e68b5667a09f66a41381251c573
Sep 11 17:15:01.152153 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:01.152108    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:01.187033 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:01.186995    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb" event={"ID":"09623531-959d-4d48-b153-4accd0a14188","Type":"ContainerStarted","Data":"3f6687312138ab0da35ee65b9b471f4f60a8104baf4d4e7cf63751c7e0c49498"}
Sep 11 17:15:01.187521 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:01.187062    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb" event={"ID":"09623531-959d-4d48-b153-4accd0a14188","Type":"ContainerStarted","Data":"d8778f97cd091b088775722832bba6bf825a3f0d1e5240ef59cdb70332095106"}
Sep 11 17:15:01.188775 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:01.188752    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n" event={"ID":"9a16751d-0d92-481b-9b41-1392035b51bd","Type":"ContainerStarted","Data":"87eebfa22bd710bab67a76b7e6ff0d250ad4ea796df6e132ce84dd9535f10c44"}
Sep 11 17:15:01.188920 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:01.188789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n" event={"ID":"9a16751d-0d92-481b-9b41-1392035b51bd","Type":"ContainerStarted","Data":"b9a97972d8fa28b8aa94d05c8122a26b2ed58e68b5667a09f66a41381251c573"}
Sep 11 17:15:01.213638 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:01.207068    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb" podStartSLOduration=1.207049317 podStartE2EDuration="1.207049317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:15:01.204789494 +0000 UTC m=+1897.612008143" watchObservedRunningTime="2026-09-11 17:15:01.207049317 +0000 UTC m=+1897.614267964"
Sep 11 17:15:01.220189 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:01.220131    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n" podStartSLOduration=1.220112248 podStartE2EDuration="1.220112248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:15:01.219226055 +0000 UTC m=+1897.626444704" watchObservedRunningTime="2026-09-11 17:15:01.220112248 +0000 UTC m=+1897.627330928"
Sep 11 17:15:01.230324 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:01.230275    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:01.397634 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:01.397592    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:02.196013 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:02.195979    2569 generic.go:356] "Generic (PLEG): container finished" podID="9a16751d-0d92-481b-9b41-1392035b51bd" containerID="87eebfa22bd710bab67a76b7e6ff0d250ad4ea796df6e132ce84dd9535f10c44" exitCode=0
Sep 11 17:15:02.196475 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:02.196091    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n" event={"ID":"9a16751d-0d92-481b-9b41-1392035b51bd","Type":"ContainerDied","Data":"87eebfa22bd710bab67a76b7e6ff0d250ad4ea796df6e132ce84dd9535f10c44"}
Sep 11 17:15:02.197675 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:02.197653    2569 generic.go:356] "Generic (PLEG): container finished" podID="09623531-959d-4d48-b153-4accd0a14188" containerID="3f6687312138ab0da35ee65b9b471f4f60a8104baf4d4e7cf63751c7e0c49498" exitCode=0
Sep 11 17:15:02.197791 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:02.197679    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb" event={"ID":"09623531-959d-4d48-b153-4accd0a14188","Type":"ContainerDied","Data":"3f6687312138ab0da35ee65b9b471f4f60a8104baf4d4e7cf63751c7e0c49498"}
Sep 11 17:15:02.297403 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:02.297373    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:03.246373 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:03.246285    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:03.362146 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:03.362118    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n"
Sep 11 17:15:03.383392 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:03.383363    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb"
Sep 11 17:15:03.965728 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:03.965690    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:04.211705 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:04.211668    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb"
Sep 11 17:15:04.211705 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:04.211696    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819115-q24tb" event={"ID":"09623531-959d-4d48-b153-4accd0a14188","Type":"ContainerDied","Data":"d8778f97cd091b088775722832bba6bf825a3f0d1e5240ef59cdb70332095106"}
Sep 11 17:15:04.212005 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:04.211755    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8778f97cd091b088775722832bba6bf825a3f0d1e5240ef59cdb70332095106"
Sep 11 17:15:04.213287 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:04.213258    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n" event={"ID":"9a16751d-0d92-481b-9b41-1392035b51bd","Type":"ContainerDied","Data":"b9a97972d8fa28b8aa94d05c8122a26b2ed58e68b5667a09f66a41381251c573"}
Sep 11 17:15:04.213392 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:04.213284    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n"
Sep 11 17:15:04.213392 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:04.213306    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9a97972d8fa28b8aa94d05c8122a26b2ed58e68b5667a09f66a41381251c573"
Sep 11 17:15:04.786929 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:04.786894    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:04.887750 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:04.887718    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:05.017670 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:05.017637    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:05.368257 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:05.368216    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:05.465527 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:05.465489    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:05.777855 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:05.777826    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:06.472862 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:06.472829    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:10.690589 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:10.690556    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:15:21.311028 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:21.310992    2569 generic.go:356] "Generic (PLEG): container finished" podID="382f4664-7c0c-427d-8263-1aac516dc26e" containerID="f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5" exitCode=0
Sep 11 17:15:21.311461 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:21.311066    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" event={"ID":"382f4664-7c0c-427d-8263-1aac516dc26e","Type":"ContainerDied","Data":"f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5"}
Sep 11 17:15:21.311580 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:21.311562    2569 scope.go:117] "RemoveContainer" containerID="f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5"
Sep 11 17:15:21.312733 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:21.312703    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:15:21.312783 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:21.312768    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:15:21.312824 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:21.312799    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:15:22.319645 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:22.319611    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" event={"ID":"382f4664-7c0c-427d-8263-1aac516dc26e","Type":"ContainerStarted","Data":"b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b"}
Sep 11 17:15:22.487235 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:22.487206    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:15:22.487464 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:22.487440    2569 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Sep 11 17:15:22.487513 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:22.487498    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Sep 11 17:15:23.487606 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:23.487562    2569 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Sep 11 17:15:23.487995 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:23.487619    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Sep 11 17:15:24.487900 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:24.487860    2569 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Sep 11 17:15:24.488415 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:24.487924    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Sep 11 17:15:25.487353 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:25.487316    2569 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Sep 11 17:15:25.487545 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:25.487373    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Sep 11 17:15:26.486816 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:26.486781    2569 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Sep 11 17:15:26.487221 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:26.486838    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Sep 11 17:15:26.926872 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:26.926839    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:27.498301 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:27.498268    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:15:27.498790 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:27.498748    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:15:27.499643 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:27.499625    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:15:27.646760 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:27.646727    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:28.350387 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:28.350353    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:29.003757 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:29.003720    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:29.849978 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:29.849940    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:30.522446 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:30.522409    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:31.271196 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:31.271154    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:32.682246 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:32.682209    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:33.570912 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:33.570875    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:34.247706 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:34.247673    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:35.080412 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:35.080378    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:35.751006 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:35.750972    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:36.420219 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:36.420177    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:43.021258 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:43.021223    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:43.118868 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:43.118831    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:43.213542 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:43.213504    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:43.275266 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:43.275178    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:43.487394 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:43.487360    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:43.595772 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:43.595687    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:44.389158 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:44.389125    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:45.091242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:45.091204    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:45.943131 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:45.943070    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:47.129421 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:47.129386    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:47.455339 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:47.455246    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:15:50.434453 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:50.434419    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:50.534046 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:50.534012    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:50.694831 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:50.694743    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"]
Sep 11 17:15:50.694982 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:50.694961    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy" containerID="cri-o://b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b" gracePeriod=30
Sep 11 17:15:50.727200 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:50.727164    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:51.800512 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:51.800478    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:51.905916 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:51.905878    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:54.464358 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:54.464316    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:55.339777 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:55.339742    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:55.953870 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:55.953846    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:15:56.024580 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.024489    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-credential-socket\") pod \"382f4664-7c0c-427d-8263-1aac516dc26e\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") "
Sep 11 17:15:56.024580 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.024532    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/382f4664-7c0c-427d-8263-1aac516dc26e-istiod-ca-cert\") pod \"382f4664-7c0c-427d-8263-1aac516dc26e\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") "
Sep 11 17:15:56.024797 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.024595    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-envoy\") pod \"382f4664-7c0c-427d-8263-1aac516dc26e\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") "
Sep 11 17:15:56.024797 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.024628    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-istio-token\") pod \"382f4664-7c0c-427d-8263-1aac516dc26e\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") "
Sep 11 17:15:56.024797 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.024657    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjzvb\" (UniqueName: \"kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-kube-api-access-bjzvb\") pod \"382f4664-7c0c-427d-8263-1aac516dc26e\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") "
Sep 11 17:15:56.024797 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.024699    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-socket\") pod \"382f4664-7c0c-427d-8263-1aac516dc26e\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") "
Sep 11 17:15:56.024797 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.024732    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/382f4664-7c0c-427d-8263-1aac516dc26e-istio-podinfo\") pod \"382f4664-7c0c-427d-8263-1aac516dc26e\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") "
Sep 11 17:15:56.024797 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.024775    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-data\") pod \"382f4664-7c0c-427d-8263-1aac516dc26e\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") "
Sep 11 17:15:56.025148 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.024810    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-certs\") pod \"382f4664-7c0c-427d-8263-1aac516dc26e\" (UID: \"382f4664-7c0c-427d-8263-1aac516dc26e\") "
Sep 11 17:15:56.025148 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.024825    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "382f4664-7c0c-427d-8263-1aac516dc26e" (UID: "382f4664-7c0c-427d-8263-1aac516dc26e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:15:56.025148 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.025059    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "382f4664-7c0c-427d-8263-1aac516dc26e" (UID: "382f4664-7c0c-427d-8263-1aac516dc26e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:15:56.025148 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.025116    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-data" (OuterVolumeSpecName: "istio-data") pod "382f4664-7c0c-427d-8263-1aac516dc26e" (UID: "382f4664-7c0c-427d-8263-1aac516dc26e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:15:56.025370 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.025245    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:15:56.025370 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.025307    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:15:56.025370 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.025310    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "382f4664-7c0c-427d-8263-1aac516dc26e" (UID: "382f4664-7c0c-427d-8263-1aac516dc26e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:15:56.025370 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.025324    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:15:56.025580 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.025396    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/382f4664-7c0c-427d-8263-1aac516dc26e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "382f4664-7c0c-427d-8263-1aac516dc26e" (UID: "382f4664-7c0c-427d-8263-1aac516dc26e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:15:56.026932 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.026903    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "382f4664-7c0c-427d-8263-1aac516dc26e" (UID: "382f4664-7c0c-427d-8263-1aac516dc26e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:15:56.027090 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.027014    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/382f4664-7c0c-427d-8263-1aac516dc26e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "382f4664-7c0c-427d-8263-1aac516dc26e" (UID: "382f4664-7c0c-427d-8263-1aac516dc26e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:15:56.027340 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.027321    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-kube-api-access-bjzvb" (OuterVolumeSpecName: "kube-api-access-bjzvb") pod "382f4664-7c0c-427d-8263-1aac516dc26e" (UID: "382f4664-7c0c-427d-8263-1aac516dc26e"). InnerVolumeSpecName "kube-api-access-bjzvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:15:56.027419 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.027334    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-istio-token" (OuterVolumeSpecName: "istio-token") pod "382f4664-7c0c-427d-8263-1aac516dc26e" (UID: "382f4664-7c0c-427d-8263-1aac516dc26e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:15:56.126509 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.126472    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/382f4664-7c0c-427d-8263-1aac516dc26e-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:15:56.126509 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.126503    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:15:56.126509 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.126512    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:15:56.126509 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.126520    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjzvb\" (UniqueName: \"kubernetes.io/projected/382f4664-7c0c-427d-8263-1aac516dc26e-kube-api-access-bjzvb\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:15:56.126783 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.126529    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/382f4664-7c0c-427d-8263-1aac516dc26e-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:15:56.126783 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.126537    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/382f4664-7c0c-427d-8263-1aac516dc26e-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:15:56.191382 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.191339    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:56.509690 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.509657    2569 generic.go:356] "Generic (PLEG): container finished" podID="382f4664-7c0c-427d-8263-1aac516dc26e" containerID="b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b" exitCode=0
Sep 11 17:15:56.509887 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.509706    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"
Sep 11 17:15:56.509887 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.509745    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" event={"ID":"382f4664-7c0c-427d-8263-1aac516dc26e","Type":"ContainerDied","Data":"b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b"}
Sep 11 17:15:56.509887 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.509807    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk" event={"ID":"382f4664-7c0c-427d-8263-1aac516dc26e","Type":"ContainerDied","Data":"a471f896b1877019528e92b9390353a5dcb3708bc4ca26fd164562c303efd622"}
Sep 11 17:15:56.509887 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.509833    2569 scope.go:117] "RemoveContainer" containerID="b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b"
Sep 11 17:15:56.521481 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.521456    2569 scope.go:117] "RemoveContainer" containerID="f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5"
Sep 11 17:15:56.529602 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.529580    2569 scope.go:117] "RemoveContainer" containerID="b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b"
Sep 11 17:15:56.529875 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:15:56.529849    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b\": container with ID starting with b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b not found: ID does not exist" containerID="b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b"
Sep 11 17:15:56.529929 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.529883    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b"} err="failed to get container status \"b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b\": rpc error: code = NotFound desc = could not find container \"b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b\": container with ID starting with b6a20ee66e45b2b3930059e33cdbe69962f53af146e366edfcd181aea29b7f1b not found: ID does not exist"
Sep 11 17:15:56.529929 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.529901    2569 scope.go:117] "RemoveContainer" containerID="f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5"
Sep 11 17:15:56.530134 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:15:56.530117    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5\": container with ID starting with f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5 not found: ID does not exist" containerID="f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5"
Sep 11 17:15:56.530190 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.530138    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5"} err="failed to get container status \"f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5\": rpc error: code = NotFound desc = could not find container \"f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5\": container with ID starting with f896061d670b346028f0b6d50d81e1accdf7ea17457f1f3088d92ae2c7ed01f5 not found: ID does not exist"
Sep 11 17:15:56.547563 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.547536    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"]
Sep 11 17:15:56.553567 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:56.553545    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891cc8-openshift-default-79f494c796-xzbrk"]
Sep 11 17:15:57.133269 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:57.133240    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:57.754302 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:57.754268    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:15:58.205452 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:58.205408    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" path="/var/lib/kubelet/pods/382f4664-7c0c-427d-8263-1aac516dc26e/volumes"
Sep 11 17:15:58.673166 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:15:58.673133    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:07.486596 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:07.486556    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:16:09.925420 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:09.925382    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:10.009450 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:10.009412    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:10.067698 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:10.067666    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:10.183320 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:10.183230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:10.248853 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:10.248811    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:10.517534 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:10.517451    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:10.596329 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:10.596295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:11.435274 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:11.435234    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:12.300630 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:12.300600    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:13.012048 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:13.012017    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:15.500238 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:15.500194    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"]
Sep 11 17:16:15.595292 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:15.595259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:20.981532 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:20.981445    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:16:24.542162 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:24.542130    2569 scope.go:117] "RemoveContainer" containerID="4f898f4af2e276ff15f1761b671bc39dadbc6fb1a6d1ce6d3a1702b02b3064d5"
Sep 11 17:16:30.520557 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.520517    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh" podUID="e38b8f95-ba39-4a2a-a157-8c790496741c" containerName="main" containerID="cri-o://e0f13c182f9c6b94b970e44939b0189fd25a4f4a84084f5973e4e458cce6a747" gracePeriod=45
Sep 11 17:16:30.690210 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.690174    2569 generic.go:356] "Generic (PLEG): container finished" podID="e38b8f95-ba39-4a2a-a157-8c790496741c" containerID="e0f13c182f9c6b94b970e44939b0189fd25a4f4a84084f5973e4e458cce6a747" exitCode=0
Sep 11 17:16:30.690398 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.690259    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh" event={"ID":"e38b8f95-ba39-4a2a-a157-8c790496741c","Type":"ContainerDied","Data":"e0f13c182f9c6b94b970e44939b0189fd25a4f4a84084f5973e4e458cce6a747"}
Sep 11 17:16:30.781891 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.781817    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:16:30.858016 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.857979    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-tmp-dir\") pod \"e38b8f95-ba39-4a2a-a157-8c790496741c\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") "
Sep 11 17:16:30.858197 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.858141    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-dshm\") pod \"e38b8f95-ba39-4a2a-a157-8c790496741c\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") "
Sep 11 17:16:30.858197 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.858186    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-model-cache\") pod \"e38b8f95-ba39-4a2a-a157-8c790496741c\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") "
Sep 11 17:16:30.858321 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.858222    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e38b8f95-ba39-4a2a-a157-8c790496741c-tls-certs\") pod \"e38b8f95-ba39-4a2a-a157-8c790496741c\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") "
Sep 11 17:16:30.858321 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.858274    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-home\") pod \"e38b8f95-ba39-4a2a-a157-8c790496741c\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") "
Sep 11 17:16:30.858427 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.858332    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnt9x\" (UniqueName: \"kubernetes.io/projected/e38b8f95-ba39-4a2a-a157-8c790496741c-kube-api-access-gnt9x\") pod \"e38b8f95-ba39-4a2a-a157-8c790496741c\" (UID: \"e38b8f95-ba39-4a2a-a157-8c790496741c\") "
Sep 11 17:16:30.858481 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.858419    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e38b8f95-ba39-4a2a-a157-8c790496741c" (UID: "e38b8f95-ba39-4a2a-a157-8c790496741c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:16:30.858570 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.858519    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-model-cache" (OuterVolumeSpecName: "model-cache") pod "e38b8f95-ba39-4a2a-a157-8c790496741c" (UID: "e38b8f95-ba39-4a2a-a157-8c790496741c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:16:30.858646 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.858602    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-home" (OuterVolumeSpecName: "home") pod "e38b8f95-ba39-4a2a-a157-8c790496741c" (UID: "e38b8f95-ba39-4a2a-a157-8c790496741c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:16:30.858774 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.858754    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:16:30.858864 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.858778    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:16:30.858864 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.858791    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:16:30.860622 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.860567    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-dshm" (OuterVolumeSpecName: "dshm") pod "e38b8f95-ba39-4a2a-a157-8c790496741c" (UID: "e38b8f95-ba39-4a2a-a157-8c790496741c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:16:30.860622 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.860569    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e38b8f95-ba39-4a2a-a157-8c790496741c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e38b8f95-ba39-4a2a-a157-8c790496741c" (UID: "e38b8f95-ba39-4a2a-a157-8c790496741c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:16:30.860818 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.860621    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e38b8f95-ba39-4a2a-a157-8c790496741c-kube-api-access-gnt9x" (OuterVolumeSpecName: "kube-api-access-gnt9x") pod "e38b8f95-ba39-4a2a-a157-8c790496741c" (UID: "e38b8f95-ba39-4a2a-a157-8c790496741c"). InnerVolumeSpecName "kube-api-access-gnt9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:16:30.960244 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.960206    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnt9x\" (UniqueName: \"kubernetes.io/projected/e38b8f95-ba39-4a2a-a157-8c790496741c-kube-api-access-gnt9x\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:16:30.960244 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.960237    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e38b8f95-ba39-4a2a-a157-8c790496741c-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:16:30.960244 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:30.960247    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e38b8f95-ba39-4a2a-a157-8c790496741c-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:16:31.697838 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:31.697805    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"
Sep 11 17:16:31.698343 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:31.697851    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh" event={"ID":"e38b8f95-ba39-4a2a-a157-8c790496741c","Type":"ContainerDied","Data":"2d1b5ced62cde41a21816eaab307a0bd0899e4f18afce0dc877c597165d734d6"}
Sep 11 17:16:31.698343 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:31.697906    2569 scope.go:117] "RemoveContainer" containerID="e0f13c182f9c6b94b970e44939b0189fd25a4f4a84084f5973e4e458cce6a747"
Sep 11 17:16:31.719654 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:31.719622    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"]
Sep 11 17:16:31.723873 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:31.723846    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-269ba8/ipp-route-269ba8-kserve-56b89fccf-nmxqh"]
Sep 11 17:16:32.206275 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:32.206242    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e38b8f95-ba39-4a2a-a157-8c790496741c" path="/var/lib/kubelet/pods/e38b8f95-ba39-4a2a-a157-8c790496741c/volumes"
Sep 11 17:16:52.528946 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:52.528911    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:53.077242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:53.077206    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:58.970341 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:58.970303    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:59.069805 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:59.069768    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:59.237573 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:59.237471    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"]
Sep 11 17:16:59.237745 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:59.237703    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy" containerID="cri-o://a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a" gracePeriod=30
Sep 11 17:16:59.260563 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:59.260532    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:16:59.983147 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:16:59.983116    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:00.908028 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:00.907994    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:01.664245 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:01.664216    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:02.474005 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:02.473964    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:03.402673 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:03.402634    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:04.494134 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.494106    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:17:04.592449 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592351    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-socket\") pod \"3b039755-032a-4ee7-889b-c3bbddf80155\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") "
Sep 11 17:17:04.592449 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592413    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b039755-032a-4ee7-889b-c3bbddf80155-istiod-ca-cert\") pod \"3b039755-032a-4ee7-889b-c3bbddf80155\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") "
Sep 11 17:17:04.592652 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592461    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-data\") pod \"3b039755-032a-4ee7-889b-c3bbddf80155\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") "
Sep 11 17:17:04.592652 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592627    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-envoy\") pod \"3b039755-032a-4ee7-889b-c3bbddf80155\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") "
Sep 11 17:17:04.592741 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592662    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-credential-socket\") pod \"3b039755-032a-4ee7-889b-c3bbddf80155\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") "
Sep 11 17:17:04.592741 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592702    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-istio-token\") pod \"3b039755-032a-4ee7-889b-c3bbddf80155\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") "
Sep 11 17:17:04.592741 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592708    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b039755-032a-4ee7-889b-c3bbddf80155" (UID: "3b039755-032a-4ee7-889b-c3bbddf80155"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:17:04.592901 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592743    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b039755-032a-4ee7-889b-c3bbddf80155" (UID: "3b039755-032a-4ee7-889b-c3bbddf80155"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:17:04.592901 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592750    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb2jc\" (UniqueName: \"kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-kube-api-access-lb2jc\") pod \"3b039755-032a-4ee7-889b-c3bbddf80155\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") "
Sep 11 17:17:04.592901 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592818    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b039755-032a-4ee7-889b-c3bbddf80155-istio-podinfo\") pod \"3b039755-032a-4ee7-889b-c3bbddf80155\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") "
Sep 11 17:17:04.592901 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592842    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b039755-032a-4ee7-889b-c3bbddf80155-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b039755-032a-4ee7-889b-c3bbddf80155" (UID: "3b039755-032a-4ee7-889b-c3bbddf80155"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:17:04.593139 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592942    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-certs\") pod \"3b039755-032a-4ee7-889b-c3bbddf80155\" (UID: \"3b039755-032a-4ee7-889b-c3bbddf80155\") "
Sep 11 17:17:04.593139 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.592962    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b039755-032a-4ee7-889b-c3bbddf80155" (UID: "3b039755-032a-4ee7-889b-c3bbddf80155"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:17:04.593281 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.593257    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b039755-032a-4ee7-889b-c3bbddf80155" (UID: "3b039755-032a-4ee7-889b-c3bbddf80155"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:17:04.593337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.593316    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:04.593337 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.593329    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:04.593452 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.593339    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:04.593452 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.593348    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b039755-032a-4ee7-889b-c3bbddf80155-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:04.594891 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.594865    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-kube-api-access-lb2jc" (OuterVolumeSpecName: "kube-api-access-lb2jc") pod "3b039755-032a-4ee7-889b-c3bbddf80155" (UID: "3b039755-032a-4ee7-889b-c3bbddf80155"). InnerVolumeSpecName "kube-api-access-lb2jc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:17:04.594996 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.594915    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b039755-032a-4ee7-889b-c3bbddf80155" (UID: "3b039755-032a-4ee7-889b-c3bbddf80155"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:17:04.595345 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.595327    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b039755-032a-4ee7-889b-c3bbddf80155-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b039755-032a-4ee7-889b-c3bbddf80155" (UID: "3b039755-032a-4ee7-889b-c3bbddf80155"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:17:04.595417 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.595342    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b039755-032a-4ee7-889b-c3bbddf80155" (UID: "3b039755-032a-4ee7-889b-c3bbddf80155"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:17:04.693840 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.693808    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:04.693840 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.693836    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb2jc\" (UniqueName: \"kubernetes.io/projected/3b039755-032a-4ee7-889b-c3bbddf80155-kube-api-access-lb2jc\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:04.693840 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.693845    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b039755-032a-4ee7-889b-c3bbddf80155-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:04.694111 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.693857    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:04.694111 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.693865    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b039755-032a-4ee7-889b-c3bbddf80155-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:04.872973 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.872884    2569 generic.go:356] "Generic (PLEG): container finished" podID="3b039755-032a-4ee7-889b-c3bbddf80155" containerID="a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a" exitCode=0
Sep 11 17:17:04.872973 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.872930    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"
Sep 11 17:17:04.873219 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.872933    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" event={"ID":"3b039755-032a-4ee7-889b-c3bbddf80155","Type":"ContainerDied","Data":"a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a"}
Sep 11 17:17:04.873219 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.873060    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78" event={"ID":"3b039755-032a-4ee7-889b-c3bbddf80155","Type":"ContainerDied","Data":"11f25986f1e5fc1e5cb5728e663ead1d03e930d44d46444502eb78cb17cdf642"}
Sep 11 17:17:04.873219 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.873129    2569 scope.go:117] "RemoveContainer" containerID="a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a"
Sep 11 17:17:04.882711 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.882690    2569 scope.go:117] "RemoveContainer" containerID="1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4"
Sep 11 17:17:04.891312 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.891290    2569 scope.go:117] "RemoveContainer" containerID="a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a"
Sep 11 17:17:04.891640 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:17:04.891619    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a\": container with ID starting with a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a not found: ID does not exist" containerID="a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a"
Sep 11 17:17:04.891697 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.891650    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a"} err="failed to get container status \"a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a\": rpc error: code = NotFound desc = could not find container \"a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a\": container with ID starting with a957702eac8d5dd0f9e4b1ef83dc06b43bc01f42423f0f5ddb8df8239f5bd47a not found: ID does not exist"
Sep 11 17:17:04.891697 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.891670    2569 scope.go:117] "RemoveContainer" containerID="1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4"
Sep 11 17:17:04.891939 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:17:04.891923    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4\": container with ID starting with 1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4 not found: ID does not exist" containerID="1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4"
Sep 11 17:17:04.891978 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.891944    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4"} err="failed to get container status \"1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4\": rpc error: code = NotFound desc = could not find container \"1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4\": container with ID starting with 1c7617d6147c42df18e492842528b76097a865c6bad375df5d6bb2dc328371c4 not found: ID does not exist"
Sep 11 17:17:04.895101 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.895056    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"]
Sep 11 17:17:04.899862 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:04.899834    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-269ba8-openshift-default-c855998c4-dpr78"]
Sep 11 17:17:06.206391 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:06.206359    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" path="/var/lib/kubelet/pods/3b039755-032a-4ee7-889b-c3bbddf80155/volumes"
Sep 11 17:17:10.227028 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.226988    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"]
Sep 11 17:17:10.227516 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.227293    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569" podUID="167913a1-309f-432c-a3be-5b7b3544c52a" containerName="maas-api" containerID="cri-o://7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15" gracePeriod=30
Sep 11 17:17:10.260878 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.260824    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:10.344303 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.344273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:10.414289 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.414256    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:10.486717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.486653    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:17:10.510550 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.510515    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:10.552457 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.552416    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kjbz\" (UniqueName: \"kubernetes.io/projected/167913a1-309f-432c-a3be-5b7b3544c52a-kube-api-access-9kjbz\") pod \"167913a1-309f-432c-a3be-5b7b3544c52a\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") "
Sep 11 17:17:10.552657 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.552464    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/167913a1-309f-432c-a3be-5b7b3544c52a-openshift-service-ca\") pod \"167913a1-309f-432c-a3be-5b7b3544c52a\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") "
Sep 11 17:17:10.552657 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.552508    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-metrics-certs\") pod \"167913a1-309f-432c-a3be-5b7b3544c52a\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") "
Sep 11 17:17:10.552657 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.552615    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-maas-api-tls\") pod \"167913a1-309f-432c-a3be-5b7b3544c52a\" (UID: \"167913a1-309f-432c-a3be-5b7b3544c52a\") "
Sep 11 17:17:10.552860 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.552836    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/167913a1-309f-432c-a3be-5b7b3544c52a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "167913a1-309f-432c-a3be-5b7b3544c52a" (UID: "167913a1-309f-432c-a3be-5b7b3544c52a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:17:10.552934 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.552919    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/167913a1-309f-432c-a3be-5b7b3544c52a-openshift-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:10.554836 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.554802    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "167913a1-309f-432c-a3be-5b7b3544c52a" (UID: "167913a1-309f-432c-a3be-5b7b3544c52a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:17:10.554943 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.554844    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "167913a1-309f-432c-a3be-5b7b3544c52a" (UID: "167913a1-309f-432c-a3be-5b7b3544c52a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:17:10.554943 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.554861    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/167913a1-309f-432c-a3be-5b7b3544c52a-kube-api-access-9kjbz" (OuterVolumeSpecName: "kube-api-access-9kjbz") pod "167913a1-309f-432c-a3be-5b7b3544c52a" (UID: "167913a1-309f-432c-a3be-5b7b3544c52a"). InnerVolumeSpecName "kube-api-access-9kjbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:17:10.571380 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.571316    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:10.653590 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.653557    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-maas-api-tls\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:10.653590 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.653585    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kjbz\" (UniqueName: \"kubernetes.io/projected/167913a1-309f-432c-a3be-5b7b3544c52a-kube-api-access-9kjbz\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:10.653590 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.653596    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/167913a1-309f-432c-a3be-5b7b3544c52a-metrics-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:10.905938 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.905907    2569 generic.go:356] "Generic (PLEG): container finished" podID="167913a1-309f-432c-a3be-5b7b3544c52a" containerID="7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15" exitCode=0
Sep 11 17:17:10.906138 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.905950    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"
Sep 11 17:17:10.906138 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.905952    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569" event={"ID":"167913a1-309f-432c-a3be-5b7b3544c52a","Type":"ContainerDied","Data":"7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15"}
Sep 11 17:17:10.906138 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.906090    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569" event={"ID":"167913a1-309f-432c-a3be-5b7b3544c52a","Type":"ContainerDied","Data":"222834f5f3e166fc95d323ed858382bb1cecc4d8878251bb4be7319a6aba60e8"}
Sep 11 17:17:10.906138 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.906119    2569 scope.go:117] "RemoveContainer" containerID="7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15"
Sep 11 17:17:10.916195 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.916177    2569 scope.go:117] "RemoveContainer" containerID="7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15"
Sep 11 17:17:10.916476 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:17:10.916458    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15\": container with ID starting with 7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15 not found: ID does not exist" containerID="7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15"
Sep 11 17:17:10.916527 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.916485    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15"} err="failed to get container status \"7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15\": rpc error: code = NotFound desc = could not find container \"7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15\": container with ID starting with 7e496584ce12cd9a1c483943f5d17e3ba09a404565168e4de8bf868e69b8fd15 not found: ID does not exist"
Sep 11 17:17:10.928576 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.928550    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"]
Sep 11 17:17:10.932840 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.932819    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-750358-65ff49769f-pm569"]
Sep 11 17:17:10.978760 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:10.978728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:11.079767 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:11.079737    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:11.790752 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:11.790723    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:12.205537 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:12.205504    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="167913a1-309f-432c-a3be-5b7b3544c52a" path="/var/lib/kubelet/pods/167913a1-309f-432c-a3be-5b7b3544c52a/volumes"
Sep 11 17:17:13.016158 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:13.016122    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:13.593713 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:13.593676    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:17:14.949019 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:14.948982    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:17.863902 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:17.863864    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:17.947525 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:17.947494    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:18.160675 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.160645    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"]
Sep 11 17:17:18.160872 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.160848    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy" containerID="cri-o://0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b" gracePeriod=30
Sep 11 17:17:18.280838 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.280794    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:18.583211 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583130    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"]
Sep 11 17:17:18.583655 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583634    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy"
Sep 11 17:17:18.583709 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583660    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy"
Sep 11 17:17:18.583709 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583673    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e38b8f95-ba39-4a2a-a157-8c790496741c" containerName="main"
Sep 11 17:17:18.583709 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583682    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e38b8f95-ba39-4a2a-a157-8c790496741c" containerName="main"
Sep 11 17:17:18.583709 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583701    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy"
Sep 11 17:17:18.583842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583710    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy"
Sep 11 17:17:18.583842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583732    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="167913a1-309f-432c-a3be-5b7b3544c52a" containerName="maas-api"
Sep 11 17:17:18.583842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583742    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="167913a1-309f-432c-a3be-5b7b3544c52a" containerName="maas-api"
Sep 11 17:17:18.583842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583753    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a16751d-0d92-481b-9b41-1392035b51bd" containerName="cleanup"
Sep 11 17:17:18.583842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583761    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a16751d-0d92-481b-9b41-1392035b51bd" containerName="cleanup"
Sep 11 17:17:18.583842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583788    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy"
Sep 11 17:17:18.583842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583796    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy"
Sep 11 17:17:18.583842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583807    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09623531-959d-4d48-b153-4accd0a14188" containerName="cleanup"
Sep 11 17:17:18.583842 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583815    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="09623531-959d-4d48-b153-4accd0a14188" containerName="cleanup"
Sep 11 17:17:18.584178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583914    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy"
Sep 11 17:17:18.584178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583926    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy"
Sep 11 17:17:18.584178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583936    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="382f4664-7c0c-427d-8263-1aac516dc26e" containerName="istio-proxy"
Sep 11 17:17:18.584178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583945    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e38b8f95-ba39-4a2a-a157-8c790496741c" containerName="main"
Sep 11 17:17:18.584178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583955    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a16751d-0d92-481b-9b41-1392035b51bd" containerName="cleanup"
Sep 11 17:17:18.584178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583964    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="167913a1-309f-432c-a3be-5b7b3544c52a" containerName="maas-api"
Sep 11 17:17:18.584178 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.583976    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="09623531-959d-4d48-b153-4accd0a14188" containerName="cleanup"
Sep 11 17:17:18.588259 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.588240    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"
Sep 11 17:17:18.601032 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.601009    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:17:18.601205 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.601051    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 17:17:18.628956 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.628923    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"]
Sep 11 17:17:18.734988 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.734955    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09b024ef-3705-45ee-9796-c00216bcd3f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp\" (UID: \"09b024ef-3705-45ee-9796-c00216bcd3f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"
Sep 11 17:17:18.748720 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.748684    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:18.836519 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.836419    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09b024ef-3705-45ee-9796-c00216bcd3f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp\" (UID: \"09b024ef-3705-45ee-9796-c00216bcd3f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"
Sep 11 17:17:18.837099 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.837047    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09b024ef-3705-45ee-9796-c00216bcd3f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp\" (UID: \"09b024ef-3705-45ee-9796-c00216bcd3f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"
Sep 11 17:17:18.897813 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:18.897775    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"
Sep 11 17:17:19.179104 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:19.177363    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"]
Sep 11 17:17:19.184741 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:17:19.184699    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b024ef_3705_45ee_9796_c00216bcd3f7.slice/crio-d2dfcd00c41c0c849ecfabca61525734a688bea01fd904698f1b7b06d9ae1c3f WatchSource:0}: Error finding container d2dfcd00c41c0c849ecfabca61525734a688bea01fd904698f1b7b06d9ae1c3f: Status 404 returned error can't find the container with id d2dfcd00c41c0c849ecfabca61525734a688bea01fd904698f1b7b06d9ae1c3f
Sep 11 17:17:19.538279 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:19.538182    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:19.637837 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:19.637797    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:19.790187 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:19.790091    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:19.958683 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:19.958652    2569 generic.go:356] "Generic (PLEG): container finished" podID="09b024ef-3705-45ee-9796-c00216bcd3f7" containerID="f169cf12baadff657c9ee52e24b30e9c33f1148487d70271bf4246617e128e8c" exitCode=0
Sep 11 17:17:19.959209 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:19.958804    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp" event={"ID":"09b024ef-3705-45ee-9796-c00216bcd3f7","Type":"ContainerDied","Data":"f169cf12baadff657c9ee52e24b30e9c33f1148487d70271bf4246617e128e8c"}
Sep 11 17:17:19.959365 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:19.959248    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp" event={"ID":"09b024ef-3705-45ee-9796-c00216bcd3f7","Type":"ContainerStarted","Data":"d2dfcd00c41c0c849ecfabca61525734a688bea01fd904698f1b7b06d9ae1c3f"}
Sep 11 17:17:20.802490 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:20.802461    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:21.095555 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:21.095524    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"
Sep 11 17:17:21.159674 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:21.159639    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09b024ef-3705-45ee-9796-c00216bcd3f7-maas-api-service-ca\") pod \"09b024ef-3705-45ee-9796-c00216bcd3f7\" (UID: \"09b024ef-3705-45ee-9796-c00216bcd3f7\") "
Sep 11 17:17:21.160039 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:21.160015    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09b024ef-3705-45ee-9796-c00216bcd3f7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "09b024ef-3705-45ee-9796-c00216bcd3f7" (UID: "09b024ef-3705-45ee-9796-c00216bcd3f7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:17:21.261168 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:21.261132    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09b024ef-3705-45ee-9796-c00216bcd3f7-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:21.576406 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:21.576367    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:21.971265 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:21.971231    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"
Sep 11 17:17:21.971441 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:21.971226    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp" event={"ID":"09b024ef-3705-45ee-9796-c00216bcd3f7","Type":"ContainerDied","Data":"d2dfcd00c41c0c849ecfabca61525734a688bea01fd904698f1b7b06d9ae1c3f"}
Sep 11 17:17:21.971441 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:21.971364    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2dfcd00c41c0c849ecfabca61525734a688bea01fd904698f1b7b06d9ae1c3f"
Sep 11 17:17:23.421419 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.421351    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:17:23.584463 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.584424    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-data\") pod \"038a0922-55f7-40c9-8445-722bb7702728\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") "
Sep 11 17:17:23.584463 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.584468    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-credential-socket\") pod \"038a0922-55f7-40c9-8445-722bb7702728\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") "
Sep 11 17:17:23.584717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.584500    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpq8j\" (UniqueName: \"kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-kube-api-access-hpq8j\") pod \"038a0922-55f7-40c9-8445-722bb7702728\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") "
Sep 11 17:17:23.584717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.584534    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-envoy\") pod \"038a0922-55f7-40c9-8445-722bb7702728\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") "
Sep 11 17:17:23.584717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.584567    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-socket\") pod \"038a0922-55f7-40c9-8445-722bb7702728\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") "
Sep 11 17:17:23.584717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.584616    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/038a0922-55f7-40c9-8445-722bb7702728-istiod-ca-cert\") pod \"038a0922-55f7-40c9-8445-722bb7702728\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") "
Sep 11 17:17:23.584717 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.584665    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/038a0922-55f7-40c9-8445-722bb7702728-istio-podinfo\") pod \"038a0922-55f7-40c9-8445-722bb7702728\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") "
Sep 11 17:17:23.584911 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.584766    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-data" (OuterVolumeSpecName: "istio-data") pod "038a0922-55f7-40c9-8445-722bb7702728" (UID: "038a0922-55f7-40c9-8445-722bb7702728"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:17:23.584911 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.584781    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "038a0922-55f7-40c9-8445-722bb7702728" (UID: "038a0922-55f7-40c9-8445-722bb7702728"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:17:23.584911 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.584771    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-istio-token\") pod \"038a0922-55f7-40c9-8445-722bb7702728\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") "
Sep 11 17:17:23.584911 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.584852    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-certs\") pod \"038a0922-55f7-40c9-8445-722bb7702728\" (UID: \"038a0922-55f7-40c9-8445-722bb7702728\") "
Sep 11 17:17:23.585241 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.585211    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/038a0922-55f7-40c9-8445-722bb7702728-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "038a0922-55f7-40c9-8445-722bb7702728" (UID: "038a0922-55f7-40c9-8445-722bb7702728"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:17:23.585241 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.585215    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "038a0922-55f7-40c9-8445-722bb7702728" (UID: "038a0922-55f7-40c9-8445-722bb7702728"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:17:23.585393 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.585257    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "038a0922-55f7-40c9-8445-722bb7702728" (UID: "038a0922-55f7-40c9-8445-722bb7702728"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:17:23.585393 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.585301    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:23.585393 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.585323    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:23.587102 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.587051    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/038a0922-55f7-40c9-8445-722bb7702728-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "038a0922-55f7-40c9-8445-722bb7702728" (UID: "038a0922-55f7-40c9-8445-722bb7702728"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:17:23.587102 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.587086    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "038a0922-55f7-40c9-8445-722bb7702728" (UID: "038a0922-55f7-40c9-8445-722bb7702728"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:17:23.587102 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.587071    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-kube-api-access-hpq8j" (OuterVolumeSpecName: "kube-api-access-hpq8j") pod "038a0922-55f7-40c9-8445-722bb7702728" (UID: "038a0922-55f7-40c9-8445-722bb7702728"). InnerVolumeSpecName "kube-api-access-hpq8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:17:23.587273 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.587106    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-istio-token" (OuterVolumeSpecName: "istio-token") pod "038a0922-55f7-40c9-8445-722bb7702728" (UID: "038a0922-55f7-40c9-8445-722bb7702728"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:17:23.686347 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.686266    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:23.686347 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.686297    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:23.686347 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.686307    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/038a0922-55f7-40c9-8445-722bb7702728-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:23.686347 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.686316    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/038a0922-55f7-40c9-8445-722bb7702728-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:23.686347 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.686325    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:23.686347 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.686333    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/038a0922-55f7-40c9-8445-722bb7702728-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:23.686347 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.686341    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpq8j\" (UniqueName: \"kubernetes.io/projected/038a0922-55f7-40c9-8445-722bb7702728-kube-api-access-hpq8j\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:23.982380 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.982292    2569 generic.go:356] "Generic (PLEG): container finished" podID="038a0922-55f7-40c9-8445-722bb7702728" containerID="0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b" exitCode=0
Sep 11 17:17:23.982380 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.982339    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"
Sep 11 17:17:23.982563 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.982339    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" event={"ID":"038a0922-55f7-40c9-8445-722bb7702728","Type":"ContainerDied","Data":"0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b"}
Sep 11 17:17:23.982563 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.982459    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd" event={"ID":"038a0922-55f7-40c9-8445-722bb7702728","Type":"ContainerDied","Data":"80aa75c94d5df910bd6d00f5c67e52fea57861bcb31b148857f04dc254c905c8"}
Sep 11 17:17:23.982563 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.982484    2569 scope.go:117] "RemoveContainer" containerID="0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b"
Sep 11 17:17:23.992506 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:23.992489    2569 scope.go:117] "RemoveContainer" containerID="1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393"
Sep 11 17:17:24.000668 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:24.000645    2569 scope.go:117] "RemoveContainer" containerID="0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b"
Sep 11 17:17:24.000940 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:17:24.000916    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b\": container with ID starting with 0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b not found: ID does not exist" containerID="0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b"
Sep 11 17:17:24.001031 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:24.000950    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b"} err="failed to get container status \"0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b\": rpc error: code = NotFound desc = could not find container \"0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b\": container with ID starting with 0c1f66bc76db1cbae9b5e35f1b09fd6bcf337694d316302e1cb9d9ef9687d73b not found: ID does not exist"
Sep 11 17:17:24.001031 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:24.000976    2569 scope.go:117] "RemoveContainer" containerID="1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393"
Sep 11 17:17:24.001286 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:17:24.001268    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393\": container with ID starting with 1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393 not found: ID does not exist" containerID="1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393"
Sep 11 17:17:24.001344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:24.001294    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393"} err="failed to get container status \"1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393\": rpc error: code = NotFound desc = could not find container \"1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393\": container with ID starting with 1f3f262576cb1ba9df0f550965821fc0e6fc75dd5c34e7a5158729d63c55a393 not found: ID does not exist"
Sep 11 17:17:24.004975 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:24.004950    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"]
Sep 11 17:17:24.009433 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:24.009412    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-750358-openshift-default-b85cb6cfb-lrsvd"]
Sep 11 17:17:24.210503 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:24.210468    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="038a0922-55f7-40c9-8445-722bb7702728" path="/var/lib/kubelet/pods/038a0922-55f7-40c9-8445-722bb7702728/volumes"
Sep 11 17:17:24.568411 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:24.568372    2569 scope.go:117] "RemoveContainer" containerID="8b25170d5b74116a8ad1a31e9dfea98309d610aa3f39ed21ff8c3b83a9ef92a1"
Sep 11 17:17:25.388360 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:25.388319    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:28.695851 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:28.695820    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:29.450334 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:29.450284    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:29.456661 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:29.456623    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n"]
Sep 11 17:17:29.459841 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:29.459818    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2fae12-29819115-9hp2n"]
Sep 11 17:17:29.536604 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:29.536569    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:29.597830 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:29.597800    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:29.698418 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:29.698381    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:29.699528 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:29.699508    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"]
Sep 11 17:17:29.705425 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:29.705356    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2fae12-sj7jp"]
Sep 11 17:17:29.754835 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:29.754808    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:30.090801 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:30.090727    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:30.181425 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:30.181393    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:30.205435 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:30.205402    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09b024ef-3705-45ee-9796-c00216bcd3f7" path="/var/lib/kubelet/pods/09b024ef-3705-45ee-9796-c00216bcd3f7/volumes"
Sep 11 17:17:30.205724 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:30.205711    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a16751d-0d92-481b-9b41-1392035b51bd" path="/var/lib/kubelet/pods/9a16751d-0d92-481b-9b41-1392035b51bd/volumes"
Sep 11 17:17:30.971001 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:30.970967    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:32.500305 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:32.500269    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:17:36.359206 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:36.359165    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:36.421182 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:36.421143    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:36.661496 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:36.661468    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:37.047795 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.047702    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"]
Sep 11 17:17:37.048111 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.048096    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy"
Sep 11 17:17:37.048161 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.048113    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy"
Sep 11 17:17:37.048161 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.048127    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy"
Sep 11 17:17:37.048161 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.048133    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy"
Sep 11 17:17:37.048161 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.048147    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09b024ef-3705-45ee-9796-c00216bcd3f7" containerName="revoke-keys"
Sep 11 17:17:37.048161 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.048152    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="09b024ef-3705-45ee-9796-c00216bcd3f7" containerName="revoke-keys"
Sep 11 17:17:37.048318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.048218    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy"
Sep 11 17:17:37.048318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.048225    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="09b024ef-3705-45ee-9796-c00216bcd3f7" containerName="revoke-keys"
Sep 11 17:17:37.048318 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.048234    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b039755-032a-4ee7-889b-c3bbddf80155" containerName="istio-proxy"
Sep 11 17:17:37.051572 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.051554    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"
Sep 11 17:17:37.053741 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.053712    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:17:37.053741 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.053713    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pc4qb\""
Sep 11 17:17:37.059524 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.059499    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"]
Sep 11 17:17:37.109957 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.109920    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30039d32-7754-4b00-9aff-ad0b52f91d16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc\" (UID: \"30039d32-7754-4b00-9aff-ad0b52f91d16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"
Sep 11 17:17:37.210547 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.210509    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30039d32-7754-4b00-9aff-ad0b52f91d16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc\" (UID: \"30039d32-7754-4b00-9aff-ad0b52f91d16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"
Sep 11 17:17:37.211208 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.211184    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30039d32-7754-4b00-9aff-ad0b52f91d16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc\" (UID: \"30039d32-7754-4b00-9aff-ad0b52f91d16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"
Sep 11 17:17:37.245580 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.245541    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:37.363167 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.363069    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"
Sep 11 17:17:37.635870 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:37.635806    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"]
Sep 11 17:17:37.638344 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:17:37.638313    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30039d32_7754_4b00_9aff_ad0b52f91d16.slice/crio-0d6c9056fc67d554f5a8cb5bb6d715b3d9a7456edd15eeb3ab6dfe097710c4db WatchSource:0}: Error finding container 0d6c9056fc67d554f5a8cb5bb6d715b3d9a7456edd15eeb3ab6dfe097710c4db: Status 404 returned error can't find the container with id 0d6c9056fc67d554f5a8cb5bb6d715b3d9a7456edd15eeb3ab6dfe097710c4db
Sep 11 17:17:38.053304 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:38.053216    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc" event={"ID":"30039d32-7754-4b00-9aff-ad0b52f91d16","Type":"ContainerStarted","Data":"14f3f30bc27e2e854a81415370a3ea4455029abeae6034e17bc66855236bddd8"}
Sep 11 17:17:38.053304 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:38.053281    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc" event={"ID":"30039d32-7754-4b00-9aff-ad0b52f91d16","Type":"ContainerStarted","Data":"0d6c9056fc67d554f5a8cb5bb6d715b3d9a7456edd15eeb3ab6dfe097710c4db"}
Sep 11 17:17:38.071938 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:38.071890    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc" podStartSLOduration=1.071871073 podStartE2EDuration="1.071871073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:17:38.071729331 +0000 UTC m=+2054.478947999" watchObservedRunningTime="2026-09-11 17:17:38.071871073 +0000 UTC m=+2054.479089721"
Sep 11 17:17:38.100457 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:38.100423    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:39.002600 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:39.002566    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:39.061292 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:39.061260    2569 generic.go:356] "Generic (PLEG): container finished" podID="30039d32-7754-4b00-9aff-ad0b52f91d16" containerID="14f3f30bc27e2e854a81415370a3ea4455029abeae6034e17bc66855236bddd8" exitCode=0
Sep 11 17:17:39.061453 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:39.061312    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc" event={"ID":"30039d32-7754-4b00-9aff-ad0b52f91d16","Type":"ContainerDied","Data":"14f3f30bc27e2e854a81415370a3ea4455029abeae6034e17bc66855236bddd8"}
Sep 11 17:17:40.202816 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:40.202785    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"
Sep 11 17:17:40.232154 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:40.232123    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30039d32-7754-4b00-9aff-ad0b52f91d16-maas-api-service-ca\") pod \"30039d32-7754-4b00-9aff-ad0b52f91d16\" (UID: \"30039d32-7754-4b00-9aff-ad0b52f91d16\") "
Sep 11 17:17:40.232470 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:40.232449    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30039d32-7754-4b00-9aff-ad0b52f91d16-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "30039d32-7754-4b00-9aff-ad0b52f91d16" (UID: "30039d32-7754-4b00-9aff-ad0b52f91d16"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:17:40.333451 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:40.333367    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30039d32-7754-4b00-9aff-ad0b52f91d16-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:17:41.075377 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:41.075341    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"
Sep 11 17:17:41.075377 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:41.075358    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc" event={"ID":"30039d32-7754-4b00-9aff-ad0b52f91d16","Type":"ContainerDied","Data":"0d6c9056fc67d554f5a8cb5bb6d715b3d9a7456edd15eeb3ab6dfe097710c4db"}
Sep 11 17:17:41.075617 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:41.075404    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d6c9056fc67d554f5a8cb5bb6d715b3d9a7456edd15eeb3ab6dfe097710c4db"
Sep 11 17:17:47.621216 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:47.621062    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:47.698430 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:47.698385    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:47.769576 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:47.769543    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:47.847182 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:47.847145    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"]
Sep 11 17:17:47.848810 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:47.848788    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-327a3a-sc4cc"]
Sep 11 17:17:47.858365 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:47.858322    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:47.921771 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:47.921737    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:48.205264 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:48.205179    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30039d32-7754-4b00-9aff-ad0b52f91d16" path="/var/lib/kubelet/pods/30039d32-7754-4b00-9aff-ad0b52f91d16/volumes"
Sep 11 17:17:48.487742 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:48.487655    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:48.557263 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:48.557229    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:50.588999 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:50.588958    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:17:54.726255 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:54.726217    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:54.784633 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:54.784602    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:55.019380 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:55.019303    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"]
Sep 11 17:17:55.019563 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:55.019539    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy" containerID="cri-o://5fcbc3070559a96617950ba71679387b53fdabde3f4e01731e6384c8d23a2cdf" gracePeriod=30
Sep 11 17:17:55.038066 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:55.038035    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:17:59.594289 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:17:59.594252    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:18:00.175732 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.175699    2569 generic.go:356] "Generic (PLEG): container finished" podID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerID="5fcbc3070559a96617950ba71679387b53fdabde3f4e01731e6384c8d23a2cdf" exitCode=0
Sep 11 17:18:00.175894 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.175769    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" event={"ID":"6008b707-b05d-4585-b319-f4cc8cf77d50","Type":"ContainerDied","Data":"5fcbc3070559a96617950ba71679387b53fdabde3f4e01731e6384c8d23a2cdf"}
Sep 11 17:18:00.175894 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.175829    2569 scope.go:117] "RemoveContainer" containerID="f1b573cb262acf4f2bc036276050337af21a1c5f655f8b41c7a226aa7487d840"
Sep 11 17:18:00.282056 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.281989    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:18:00.416334 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.416298    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-socket\") pod \"6008b707-b05d-4585-b319-f4cc8cf77d50\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") "
Sep 11 17:18:00.416502 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.416341    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzxrh\" (UniqueName: \"kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-kube-api-access-nzxrh\") pod \"6008b707-b05d-4585-b319-f4cc8cf77d50\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") "
Sep 11 17:18:00.416502 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.416364    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-podinfo\") pod \"6008b707-b05d-4585-b319-f4cc8cf77d50\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") "
Sep 11 17:18:00.416502 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.416385    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-credential-socket\") pod \"6008b707-b05d-4585-b319-f4cc8cf77d50\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") "
Sep 11 17:18:00.416502 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.416451    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-token\") pod \"6008b707-b05d-4585-b319-f4cc8cf77d50\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") "
Sep 11 17:18:00.416713 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.416551    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-envoy\") pod \"6008b707-b05d-4585-b319-f4cc8cf77d50\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") "
Sep 11 17:18:00.416713 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.416585    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6008b707-b05d-4585-b319-f4cc8cf77d50-istiod-ca-cert\") pod \"6008b707-b05d-4585-b319-f4cc8cf77d50\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") "
Sep 11 17:18:00.416713 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.416607    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-certs\") pod \"6008b707-b05d-4585-b319-f4cc8cf77d50\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") "
Sep 11 17:18:00.416713 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.416632    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-data\") pod \"6008b707-b05d-4585-b319-f4cc8cf77d50\" (UID: \"6008b707-b05d-4585-b319-f4cc8cf77d50\") "
Sep 11 17:18:00.416923 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.416686    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6008b707-b05d-4585-b319-f4cc8cf77d50" (UID: "6008b707-b05d-4585-b319-f4cc8cf77d50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:18:00.416923 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.416769    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6008b707-b05d-4585-b319-f4cc8cf77d50" (UID: "6008b707-b05d-4585-b319-f4cc8cf77d50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:18:00.417050 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.417016    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:18:00.417050 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.417039    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:18:00.417223 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.417193    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6008b707-b05d-4585-b319-f4cc8cf77d50-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6008b707-b05d-4585-b319-f4cc8cf77d50" (UID: "6008b707-b05d-4585-b319-f4cc8cf77d50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:18:00.417470 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.417438    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-data" (OuterVolumeSpecName: "istio-data") pod "6008b707-b05d-4585-b319-f4cc8cf77d50" (UID: "6008b707-b05d-4585-b319-f4cc8cf77d50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:18:00.417583 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.417464    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6008b707-b05d-4585-b319-f4cc8cf77d50" (UID: "6008b707-b05d-4585-b319-f4cc8cf77d50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:18:00.418755 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.418725    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-kube-api-access-nzxrh" (OuterVolumeSpecName: "kube-api-access-nzxrh") pod "6008b707-b05d-4585-b319-f4cc8cf77d50" (UID: "6008b707-b05d-4585-b319-f4cc8cf77d50"). InnerVolumeSpecName "kube-api-access-nzxrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:18:00.418857 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.418775    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-token" (OuterVolumeSpecName: "istio-token") pod "6008b707-b05d-4585-b319-f4cc8cf77d50" (UID: "6008b707-b05d-4585-b319-f4cc8cf77d50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:18:00.419014 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.418997    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6008b707-b05d-4585-b319-f4cc8cf77d50" (UID: "6008b707-b05d-4585-b319-f4cc8cf77d50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:18:00.419206 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.419191    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6008b707-b05d-4585-b319-f4cc8cf77d50" (UID: "6008b707-b05d-4585-b319-f4cc8cf77d50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:18:00.517859 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.517829    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:18:00.517859 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.517853    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:18:00.517859 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.517862    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:18:00.518101 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.517891    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6008b707-b05d-4585-b319-f4cc8cf77d50-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:18:00.518101 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.517899    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:18:00.518101 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.517908    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6008b707-b05d-4585-b319-f4cc8cf77d50-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:18:00.518101 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:00.517916    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzxrh\" (UniqueName: \"kubernetes.io/projected/6008b707-b05d-4585-b319-f4cc8cf77d50-kube-api-access-nzxrh\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:18:01.181372 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:01.181333    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"
Sep 11 17:18:01.181902 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:01.181367    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq" event={"ID":"6008b707-b05d-4585-b319-f4cc8cf77d50","Type":"ContainerDied","Data":"64fb114520bf03d3062f3832389c96f2cb75f0a8bfa5c5fdcb713f1dd4d9a6d7"}
Sep 11 17:18:01.181902 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:01.181440    2569 scope.go:117] "RemoveContainer" containerID="5fcbc3070559a96617950ba71679387b53fdabde3f4e01731e6384c8d23a2cdf"
Sep 11 17:18:01.214130 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:01.213671    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"]
Sep 11 17:18:01.218917 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:01.218889    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-327a3a-openshift-default-5b79db7d64-ghqgq"]
Sep 11 17:18:02.205138 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:02.205056    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" path="/var/lib/kubelet/pods/6008b707-b05d-4585-b319-f4cc8cf77d50/volumes"
Sep 11 17:18:37.946438 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:18:37.946399    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:00.910601 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:19:00.910566    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:20.684434 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:19:20.684361    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:19:21.588274 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:19:21.588236    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:23.170977 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:19:23.170938    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:38.228505 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:19:38.228469    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:47.493372 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:19:47.493336    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:19:54.985586 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:19:54.985547    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:20:05.147923 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:20:05.147888    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:20:23.791049 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:20:23.791008    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:20:29.783835 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:20:29.783797    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:21:24.631341 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:21:24.631304    2569 scope.go:117] "RemoveContainer" containerID="87eebfa22bd710bab67a76b7e6ff0d250ad4ea796df6e132ce84dd9535f10c44"
Sep 11 17:21:56.575705 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:21:56.575668    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:22:00.618048 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:00.617994    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:05.689218 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:05.689182    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:22:21.385562 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.385480    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"]
Sep 11 17:22:21.385986 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.385885    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30039d32-7754-4b00-9aff-ad0b52f91d16" containerName="revoke-keys"
Sep 11 17:22:21.385986 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.385901    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="30039d32-7754-4b00-9aff-ad0b52f91d16" containerName="revoke-keys"
Sep 11 17:22:21.385986 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.385916    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy"
Sep 11 17:22:21.385986 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.385921    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy"
Sep 11 17:22:21.385986 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.385932    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy"
Sep 11 17:22:21.385986 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.385938    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy"
Sep 11 17:22:21.385986 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.385955    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy"
Sep 11 17:22:21.385986 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.385960    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy"
Sep 11 17:22:21.386290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.386032    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy"
Sep 11 17:22:21.386290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.386043    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="30039d32-7754-4b00-9aff-ad0b52f91d16" containerName="revoke-keys"
Sep 11 17:22:21.386290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.386051    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="038a0922-55f7-40c9-8445-722bb7702728" containerName="istio-proxy"
Sep 11 17:22:21.389182 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.389164    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.391054 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.391036    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-cb7d7a21-kserve-self-signed-certs\""
Sep 11 17:22:21.398882 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.398861    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"]
Sep 11 17:22:21.550241 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.550202    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwrm2\" (UniqueName: \"kubernetes.io/projected/541edb35-2990-4f94-81b5-77ff13c8f21d-kube-api-access-mwrm2\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.550435 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.550257    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541edb35-2990-4f94-81b5-77ff13c8f21d-tls-certs\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.550435 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.550327    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-model-cache\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.550435 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.550355    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-dshm\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.550435 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.550377    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-tmp-dir\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.550435 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.550410    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-home\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.572358 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.572321    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:21.651778 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.651682    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwrm2\" (UniqueName: \"kubernetes.io/projected/541edb35-2990-4f94-81b5-77ff13c8f21d-kube-api-access-mwrm2\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.651778 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.651750    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541edb35-2990-4f94-81b5-77ff13c8f21d-tls-certs\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.652008 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.651814    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-model-cache\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.652008 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.651841    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-dshm\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.652008 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.651867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-tmp-dir\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.652008 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.651906    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-home\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.652378 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.652347    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-home\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.652506 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.652360    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-model-cache\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.652506 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.652432    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-tmp-dir\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.654295 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.654269    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-dshm\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.654470 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.654451    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541edb35-2990-4f94-81b5-77ff13c8f21d-tls-certs\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.659046 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.659024    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwrm2\" (UniqueName: \"kubernetes.io/projected/541edb35-2990-4f94-81b5-77ff13c8f21d-kube-api-access-mwrm2\") pod \"e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") " pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.698623 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.698580    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:21.974982 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:21.974929    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"]
Sep 11 17:22:21.977286 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:22:21.977258    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod541edb35_2990_4f94_81b5_77ff13c8f21d.slice/crio-983665ae7377c30b7fb111dc5f13548aa67a568600f3c7b3abd6ab1c4127793b WatchSource:0}: Error finding container 983665ae7377c30b7fb111dc5f13548aa67a568600f3c7b3abd6ab1c4127793b: Status 404 returned error can't find the container with id 983665ae7377c30b7fb111dc5f13548aa67a568600f3c7b3abd6ab1c4127793b
Sep 11 17:22:22.490020 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:22.489981    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25" event={"ID":"541edb35-2990-4f94-81b5-77ff13c8f21d","Type":"ContainerStarted","Data":"a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773"}
Sep 11 17:22:22.490421 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:22.490033    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25" event={"ID":"541edb35-2990-4f94-81b5-77ff13c8f21d","Type":"ContainerStarted","Data":"983665ae7377c30b7fb111dc5f13548aa67a568600f3c7b3abd6ab1c4127793b"}
Sep 11 17:22:22.507196 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:22.507146    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25" podStartSLOduration=1.507129483 podStartE2EDuration="1.507129483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:22:22.506476863 +0000 UTC m=+2338.913695536" watchObservedRunningTime="2026-09-11 17:22:22.507129483 +0000 UTC m=+2338.914348130"
Sep 11 17:22:24.179674 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:24.179641    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:22:31.699329 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:31.699283    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:31.699329 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:31.699343    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:31.711967 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:31.711931    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:32.549603 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:32.549571    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:22:37.474666 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.474628    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"]
Sep 11 17:22:37.475136 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.475121    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6008b707-b05d-4585-b319-f4cc8cf77d50" containerName="istio-proxy"
Sep 11 17:22:37.478323 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.478301    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.480529 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.480507    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-cb7d7a21-kserve-self-signed-certs\""
Sep 11 17:22:37.487514 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.487483    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"]
Sep 11 17:22:37.493613 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.493576    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:37.501798 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.501763    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4wv6\" (UniqueName: \"kubernetes.io/projected/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-kube-api-access-x4wv6\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.501947 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.501824    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-home\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.501991 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.501951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tmp-dir\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.502031 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.501998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-dshm\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.502064 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.502037    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-model-cache\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.502129 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.502117    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tls-certs\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.603620 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.603569    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-model-cache\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.603801 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.603635    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tls-certs\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.603801 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.603673    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4wv6\" (UniqueName: \"kubernetes.io/projected/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-kube-api-access-x4wv6\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.603876 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.603854    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-home\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.603978 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.603962    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tmp-dir\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.604047 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.604003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-dshm\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.604134 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.604069    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-model-cache\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.604264 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.604236    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-home\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.604417 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.604286    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tmp-dir\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.606369 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.606341    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-dshm\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.606556 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.606539    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tls-certs\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.612427 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.612410    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4wv6\" (UniqueName: \"kubernetes.io/projected/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-kube-api-access-x4wv6\") pod \"e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") " pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:37.788363 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:37.788258    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:38.062719 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:38.062690    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"]
Sep 11 17:22:38.065345 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:22:38.065322    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5d28493_f486_4cbf_8c9d_b4019d1a7a56.slice/crio-348a249312645d7846b6c0bbd9d5ac2dab86c681ce2ea05821d90b6fe58c7ea7 WatchSource:0}: Error finding container 348a249312645d7846b6c0bbd9d5ac2dab86c681ce2ea05821d90b6fe58c7ea7: Status 404 returned error can't find the container with id 348a249312645d7846b6c0bbd9d5ac2dab86c681ce2ea05821d90b6fe58c7ea7
Sep 11 17:22:38.570374 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:38.570334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx" event={"ID":"d5d28493-f486-4cbf-8c9d-b4019d1a7a56","Type":"ContainerStarted","Data":"4817089075a6af64245e12d7a56aaa24ab4449be16270fea88e21c777f99bb71"}
Sep 11 17:22:38.570374 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:38.570379    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx" event={"ID":"d5d28493-f486-4cbf-8c9d-b4019d1a7a56","Type":"ContainerStarted","Data":"348a249312645d7846b6c0bbd9d5ac2dab86c681ce2ea05821d90b6fe58c7ea7"}
Sep 11 17:22:38.588808 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:38.588752    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx" podStartSLOduration=1.588735426 podStartE2EDuration="1.588735426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:22:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:22:38.587662354 +0000 UTC m=+2354.994881017" watchObservedRunningTime="2026-09-11 17:22:38.588735426 +0000 UTC m=+2354.995954074"
Sep 11 17:22:40.287506 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:40.287469    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:22:47.788710 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:47.788673    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:47.789204 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:47.789021    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:47.801455 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:47.801421    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:22:48.633134 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:22:48.633098    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:23:02.314586 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:02.314543    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:24.659323 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:24.659287    2569 scope.go:117] "RemoveContainer" containerID="f169cf12baadff657c9ee52e24b30e9c33f1148487d70271bf4246617e128e8c"
Sep 11 17:23:27.022385 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:27.022347    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"]
Sep 11 17:23:27.036766 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:27.036734    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:28.759270 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:28.759237    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"]
Sep 11 17:23:28.780406 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:28.780373    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:29.279345 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:29.279305    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:23:36.796094 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:36.796038    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:23:42.041672 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.041623    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25" podUID="541edb35-2990-4f94-81b5-77ff13c8f21d" containerName="main" containerID="cri-o://a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773" gracePeriod=45
Sep 11 17:23:42.295243 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.295185    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:23:42.421155 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.421119    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwrm2\" (UniqueName: \"kubernetes.io/projected/541edb35-2990-4f94-81b5-77ff13c8f21d-kube-api-access-mwrm2\") pod \"541edb35-2990-4f94-81b5-77ff13c8f21d\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") "
Sep 11 17:23:42.421335 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.421183    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-home\") pod \"541edb35-2990-4f94-81b5-77ff13c8f21d\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") "
Sep 11 17:23:42.421335 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.421225    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-model-cache\") pod \"541edb35-2990-4f94-81b5-77ff13c8f21d\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") "
Sep 11 17:23:42.421335 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.421283    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-dshm\") pod \"541edb35-2990-4f94-81b5-77ff13c8f21d\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") "
Sep 11 17:23:42.421335 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.421301    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541edb35-2990-4f94-81b5-77ff13c8f21d-tls-certs\") pod \"541edb35-2990-4f94-81b5-77ff13c8f21d\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") "
Sep 11 17:23:42.421335 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.421336    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-tmp-dir\") pod \"541edb35-2990-4f94-81b5-77ff13c8f21d\" (UID: \"541edb35-2990-4f94-81b5-77ff13c8f21d\") "
Sep 11 17:23:42.421621 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.421485    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-model-cache" (OuterVolumeSpecName: "model-cache") pod "541edb35-2990-4f94-81b5-77ff13c8f21d" (UID: "541edb35-2990-4f94-81b5-77ff13c8f21d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:42.421621 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.421497    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-home" (OuterVolumeSpecName: "home") pod "541edb35-2990-4f94-81b5-77ff13c8f21d" (UID: "541edb35-2990-4f94-81b5-77ff13c8f21d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:42.421621 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.421584    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:42.421745 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.421679    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "541edb35-2990-4f94-81b5-77ff13c8f21d" (UID: "541edb35-2990-4f94-81b5-77ff13c8f21d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:42.423547 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.423518    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/541edb35-2990-4f94-81b5-77ff13c8f21d-kube-api-access-mwrm2" (OuterVolumeSpecName: "kube-api-access-mwrm2") pod "541edb35-2990-4f94-81b5-77ff13c8f21d" (UID: "541edb35-2990-4f94-81b5-77ff13c8f21d"). InnerVolumeSpecName "kube-api-access-mwrm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:23:42.423656 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.423541    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-dshm" (OuterVolumeSpecName: "dshm") pod "541edb35-2990-4f94-81b5-77ff13c8f21d" (UID: "541edb35-2990-4f94-81b5-77ff13c8f21d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:42.423656 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.423542    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/541edb35-2990-4f94-81b5-77ff13c8f21d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "541edb35-2990-4f94-81b5-77ff13c8f21d" (UID: "541edb35-2990-4f94-81b5-77ff13c8f21d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:23:42.522350 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.522313    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:42.522350 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.522343    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:42.522350 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.522356    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541edb35-2990-4f94-81b5-77ff13c8f21d-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:42.522588 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.522368    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541edb35-2990-4f94-81b5-77ff13c8f21d-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:42.522588 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.522380    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwrm2\" (UniqueName: \"kubernetes.io/projected/541edb35-2990-4f94-81b5-77ff13c8f21d-kube-api-access-mwrm2\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:42.893846 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.893812    2569 generic.go:356] "Generic (PLEG): container finished" podID="541edb35-2990-4f94-81b5-77ff13c8f21d" containerID="a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773" exitCode=0
Sep 11 17:23:42.894037 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.893854    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"
Sep 11 17:23:42.894037 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.893892    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25" event={"ID":"541edb35-2990-4f94-81b5-77ff13c8f21d","Type":"ContainerDied","Data":"a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773"}
Sep 11 17:23:42.894037 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.893935    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25" event={"ID":"541edb35-2990-4f94-81b5-77ff13c8f21d","Type":"ContainerDied","Data":"983665ae7377c30b7fb111dc5f13548aa67a568600f3c7b3abd6ab1c4127793b"}
Sep 11 17:23:42.894037 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.893953    2569 scope.go:117] "RemoveContainer" containerID="a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773"
Sep 11 17:23:42.903347 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.903329    2569 scope.go:117] "RemoveContainer" containerID="a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773"
Sep 11 17:23:42.903616 ip-10-0-135-55 kubenswrapper[2569]: E0911 17:23:42.903597    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773\": container with ID starting with a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773 not found: ID does not exist" containerID="a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773"
Sep 11 17:23:42.903662 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.903624    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773"} err="failed to get container status \"a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773\": rpc error: code = NotFound desc = could not find container \"a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773\": container with ID starting with a0a5018a3cb64e9b852a26ea6194705aae88d5238db5bd9a0fc0b844e7da2773 not found: ID does not exist"
Sep 11 17:23:42.923837 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.923809    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"]
Sep 11 17:23:42.928295 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:42.928269    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-cb7d7a21-kserve-5c5cdb4787-dcd25"]
Sep 11 17:23:43.778824 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:43.778773    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx" podUID="d5d28493-f486-4cbf-8c9d-b4019d1a7a56" containerName="main" containerID="cri-o://4817089075a6af64245e12d7a56aaa24ab4449be16270fea88e21c777f99bb71" gracePeriod=45
Sep 11 17:23:43.901853 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:43.901826    2569 generic.go:356] "Generic (PLEG): container finished" podID="d5d28493-f486-4cbf-8c9d-b4019d1a7a56" containerID="4817089075a6af64245e12d7a56aaa24ab4449be16270fea88e21c777f99bb71" exitCode=0
Sep 11 17:23:43.901980 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:43.901890    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx" event={"ID":"d5d28493-f486-4cbf-8c9d-b4019d1a7a56","Type":"ContainerDied","Data":"4817089075a6af64245e12d7a56aaa24ab4449be16270fea88e21c777f99bb71"}
Sep 11 17:23:44.031502 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.031441    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:23:44.136859 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.136826    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-model-cache\") pod \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") "
Sep 11 17:23:44.137014 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.136919    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-home\") pod \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") "
Sep 11 17:23:44.137014 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.136962    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4wv6\" (UniqueName: \"kubernetes.io/projected/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-kube-api-access-x4wv6\") pod \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") "
Sep 11 17:23:44.137014 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.137011    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tls-certs\") pod \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") "
Sep 11 17:23:44.137228 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.137044    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tmp-dir\") pod \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") "
Sep 11 17:23:44.137228 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.137138    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-dshm\") pod \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\" (UID: \"d5d28493-f486-4cbf-8c9d-b4019d1a7a56\") "
Sep 11 17:23:44.137228 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.137159    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-model-cache" (OuterVolumeSpecName: "model-cache") pod "d5d28493-f486-4cbf-8c9d-b4019d1a7a56" (UID: "d5d28493-f486-4cbf-8c9d-b4019d1a7a56"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:44.137228 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.137204    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-home" (OuterVolumeSpecName: "home") pod "d5d28493-f486-4cbf-8c9d-b4019d1a7a56" (UID: "d5d28493-f486-4cbf-8c9d-b4019d1a7a56"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:44.137453 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.137330    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d5d28493-f486-4cbf-8c9d-b4019d1a7a56" (UID: "d5d28493-f486-4cbf-8c9d-b4019d1a7a56"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:44.137453 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.137432    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:44.137453 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.137448    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:44.137576 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.137461    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:44.139306 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.139273    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-kube-api-access-x4wv6" (OuterVolumeSpecName: "kube-api-access-x4wv6") pod "d5d28493-f486-4cbf-8c9d-b4019d1a7a56" (UID: "d5d28493-f486-4cbf-8c9d-b4019d1a7a56"). InnerVolumeSpecName "kube-api-access-x4wv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:23:44.139413 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.139328    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d5d28493-f486-4cbf-8c9d-b4019d1a7a56" (UID: "d5d28493-f486-4cbf-8c9d-b4019d1a7a56"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:23:44.139413 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.139338    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-dshm" (OuterVolumeSpecName: "dshm") pod "d5d28493-f486-4cbf-8c9d-b4019d1a7a56" (UID: "d5d28493-f486-4cbf-8c9d-b4019d1a7a56"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:44.204569 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.204537    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="541edb35-2990-4f94-81b5-77ff13c8f21d" path="/var/lib/kubelet/pods/541edb35-2990-4f94-81b5-77ff13c8f21d/volumes"
Sep 11 17:23:44.238040 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.238007    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4wv6\" (UniqueName: \"kubernetes.io/projected/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-kube-api-access-x4wv6\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:44.238040 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.238032    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:44.238236 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.238048    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d28493-f486-4cbf-8c9d-b4019d1a7a56-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:44.909668 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.909631    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"
Sep 11 17:23:44.910202 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.909672    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx" event={"ID":"d5d28493-f486-4cbf-8c9d-b4019d1a7a56","Type":"ContainerDied","Data":"348a249312645d7846b6c0bbd9d5ac2dab86c681ce2ea05821d90b6fe58c7ea7"}
Sep 11 17:23:44.910202 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.909723    2569 scope.go:117] "RemoveContainer" containerID="4817089075a6af64245e12d7a56aaa24ab4449be16270fea88e21c777f99bb71"
Sep 11 17:23:44.931787 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.931757    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"]
Sep 11 17:23:44.936414 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:44.936393    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-cb7d7a21-kserve-7ffdcf59f9-vnhbx"]
Sep 11 17:23:45.025447 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.025403    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:23:45.083002 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.082961    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"]
Sep 11 17:23:45.083433 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.083413    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="541edb35-2990-4f94-81b5-77ff13c8f21d" containerName="main"
Sep 11 17:23:45.083526 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.083435    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="541edb35-2990-4f94-81b5-77ff13c8f21d" containerName="main"
Sep 11 17:23:45.083526 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.083457    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5d28493-f486-4cbf-8c9d-b4019d1a7a56" containerName="main"
Sep 11 17:23:45.083526 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.083465    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5d28493-f486-4cbf-8c9d-b4019d1a7a56" containerName="main"
Sep 11 17:23:45.083685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.083562    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="541edb35-2990-4f94-81b5-77ff13c8f21d" containerName="main"
Sep 11 17:23:45.083685 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.083581    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5d28493-f486-4cbf-8c9d-b4019d1a7a56" containerName="main"
Sep 11 17:23:45.087999 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.087978    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.092602 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.092583    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:23:45.093098 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.093051    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 17:23:45.093098 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.093066    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 17:23:45.093310 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.093112    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 17:23:45.093310 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.093058    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8lxr4\""
Sep 11 17:23:45.103192 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.103169    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"]
Sep 11 17:23:45.246213 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.246113    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-metrics-certs\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.246213 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.246188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0452955-8e92-49a5-bae2-0a05e835ea9b-openshift-service-ca\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.246213 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.246210    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xbr2\" (UniqueName: \"kubernetes.io/projected/d0452955-8e92-49a5-bae2-0a05e835ea9b-kube-api-access-7xbr2\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.246442 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.246256    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-maas-api-tls\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.347753 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.347723    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0452955-8e92-49a5-bae2-0a05e835ea9b-openshift-service-ca\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.347894 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.347758    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xbr2\" (UniqueName: \"kubernetes.io/projected/d0452955-8e92-49a5-bae2-0a05e835ea9b-kube-api-access-7xbr2\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.347894 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.347805    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-maas-api-tls\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.347991 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.347893    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-metrics-certs\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.348372 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.348353    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0452955-8e92-49a5-bae2-0a05e835ea9b-openshift-service-ca\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.350486 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.350464    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-maas-api-tls\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.350586 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.350514    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-metrics-certs\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.355289 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.355267    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xbr2\" (UniqueName: \"kubernetes.io/projected/d0452955-8e92-49a5-bae2-0a05e835ea9b-kube-api-access-7xbr2\") pod \"maas-api-67d5b455cb-j422z\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") " pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.398479 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.398444    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:45.673837 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.673809    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"]
Sep 11 17:23:45.675894 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:23:45.675866    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0452955_8e92_49a5_bae2_0a05e835ea9b.slice/crio-e2b76f378bccc7bb219bc82dec50ad3759b48381c78de4eabe232e03d8126407 WatchSource:0}: Error finding container e2b76f378bccc7bb219bc82dec50ad3759b48381c78de4eabe232e03d8126407: Status 404 returned error can't find the container with id e2b76f378bccc7bb219bc82dec50ad3759b48381c78de4eabe232e03d8126407
Sep 11 17:23:45.677277 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.677258    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:23:45.916719 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:45.916679    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z" event={"ID":"d0452955-8e92-49a5-bae2-0a05e835ea9b","Type":"ContainerStarted","Data":"e2b76f378bccc7bb219bc82dec50ad3759b48381c78de4eabe232e03d8126407"}
Sep 11 17:23:46.205314 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:46.205281    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5d28493-f486-4cbf-8c9d-b4019d1a7a56" path="/var/lib/kubelet/pods/d5d28493-f486-4cbf-8c9d-b4019d1a7a56/volumes"
Sep 11 17:23:46.925184 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:46.925136    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z" event={"ID":"d0452955-8e92-49a5-bae2-0a05e835ea9b","Type":"ContainerStarted","Data":"4bc7e718a03f9f4a2c29adaec0c232427bc3bd138a705fef001412f44b5146ae"}
Sep 11 17:23:46.925653 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:46.925199    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:46.942473 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:46.942418    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z" podStartSLOduration=1.522980086 podStartE2EDuration="1.942397862s" totalImagesPullingTime="419.417776ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:23:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:23:46.940833679 +0000 UTC m=+2423.348052348" watchObservedRunningTime="2026-09-11 17:23:46.942397862 +0000 UTC m=+2423.349616551"
Sep 11 17:23:52.582142 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.580837    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:23:52.744550 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.744510    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"]
Sep 11 17:23:52.744751 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.744712    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z" podUID="d0452955-8e92-49a5-bae2-0a05e835ea9b" containerName="maas-api" containerID="cri-o://4bc7e718a03f9f4a2c29adaec0c232427bc3bd138a705fef001412f44b5146ae" gracePeriod=30
Sep 11 17:23:52.749926 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.749903    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:52.782815 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.782783    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"]
Sep 11 17:23:52.786412 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.786384    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.798038 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.798011    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"]
Sep 11 17:23:52.815540 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.815490    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2660466-8270-4504-bc69-a8477d77782f-maas-api-tls\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.815720 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.815545    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptcp9\" (UniqueName: \"kubernetes.io/projected/a2660466-8270-4504-bc69-a8477d77782f-kube-api-access-ptcp9\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.815796 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.815709    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2660466-8270-4504-bc69-a8477d77782f-metrics-certs\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.815853 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.815788    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2660466-8270-4504-bc69-a8477d77782f-openshift-service-ca\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.916622 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.916589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2660466-8270-4504-bc69-a8477d77782f-maas-api-tls\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.916781 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.916641    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptcp9\" (UniqueName: \"kubernetes.io/projected/a2660466-8270-4504-bc69-a8477d77782f-kube-api-access-ptcp9\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.916781 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.916686    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2660466-8270-4504-bc69-a8477d77782f-metrics-certs\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.916781 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.916712    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2660466-8270-4504-bc69-a8477d77782f-openshift-service-ca\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.917720 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.917668    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2660466-8270-4504-bc69-a8477d77782f-openshift-service-ca\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.919879 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.919852    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2660466-8270-4504-bc69-a8477d77782f-maas-api-tls\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.920143 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.920122    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2660466-8270-4504-bc69-a8477d77782f-metrics-certs\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.927546 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.927512    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptcp9\" (UniqueName: \"kubernetes.io/projected/a2660466-8270-4504-bc69-a8477d77782f-kube-api-access-ptcp9\") pod \"maas-api-bf6cf69f6-tws2f\" (UID: \"a2660466-8270-4504-bc69-a8477d77782f\") " pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:52.956973 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.956935    2569 generic.go:356] "Generic (PLEG): container finished" podID="d0452955-8e92-49a5-bae2-0a05e835ea9b" containerID="4bc7e718a03f9f4a2c29adaec0c232427bc3bd138a705fef001412f44b5146ae" exitCode=0
Sep 11 17:23:52.957161 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.957006    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z" event={"ID":"d0452955-8e92-49a5-bae2-0a05e835ea9b","Type":"ContainerDied","Data":"4bc7e718a03f9f4a2c29adaec0c232427bc3bd138a705fef001412f44b5146ae"}
Sep 11 17:23:52.998267 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:52.998241    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:53.017782 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.017753    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0452955-8e92-49a5-bae2-0a05e835ea9b-openshift-service-ca\") pod \"d0452955-8e92-49a5-bae2-0a05e835ea9b\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") "
Sep 11 17:23:53.017911 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.017793    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-metrics-certs\") pod \"d0452955-8e92-49a5-bae2-0a05e835ea9b\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") "
Sep 11 17:23:53.017911 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.017834    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-maas-api-tls\") pod \"d0452955-8e92-49a5-bae2-0a05e835ea9b\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") "
Sep 11 17:23:53.018046 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.017969    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xbr2\" (UniqueName: \"kubernetes.io/projected/d0452955-8e92-49a5-bae2-0a05e835ea9b-kube-api-access-7xbr2\") pod \"d0452955-8e92-49a5-bae2-0a05e835ea9b\" (UID: \"d0452955-8e92-49a5-bae2-0a05e835ea9b\") "
Sep 11 17:23:53.018210 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.018179    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0452955-8e92-49a5-bae2-0a05e835ea9b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d0452955-8e92-49a5-bae2-0a05e835ea9b" (UID: "d0452955-8e92-49a5-bae2-0a05e835ea9b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:23:53.018377 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.018355    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0452955-8e92-49a5-bae2-0a05e835ea9b-openshift-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:53.020122 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.020055    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0452955-8e92-49a5-bae2-0a05e835ea9b-kube-api-access-7xbr2" (OuterVolumeSpecName: "kube-api-access-7xbr2") pod "d0452955-8e92-49a5-bae2-0a05e835ea9b" (UID: "d0452955-8e92-49a5-bae2-0a05e835ea9b"). InnerVolumeSpecName "kube-api-access-7xbr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:23:53.020521 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.020497    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d0452955-8e92-49a5-bae2-0a05e835ea9b" (UID: "d0452955-8e92-49a5-bae2-0a05e835ea9b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:23:53.020583 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.020504    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d0452955-8e92-49a5-bae2-0a05e835ea9b" (UID: "d0452955-8e92-49a5-bae2-0a05e835ea9b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:23:53.118984 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.118890    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xbr2\" (UniqueName: \"kubernetes.io/projected/d0452955-8e92-49a5-bae2-0a05e835ea9b-kube-api-access-7xbr2\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:53.118984 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.118924    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-metrics-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:53.118984 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.118937    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0452955-8e92-49a5-bae2-0a05e835ea9b-maas-api-tls\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Sep 11 17:23:53.121836 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.121797    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:53.398114 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.398063    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"]
Sep 11 17:23:53.400404 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:23:53.400377    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2660466_8270_4504_bc69_a8477d77782f.slice/crio-0d384c809ace445fe847debd9d3d7f79e78cd9e4e2fc9d8dc97c07566313b10e WatchSource:0}: Error finding container 0d384c809ace445fe847debd9d3d7f79e78cd9e4e2fc9d8dc97c07566313b10e: Status 404 returned error can't find the container with id 0d384c809ace445fe847debd9d3d7f79e78cd9e4e2fc9d8dc97c07566313b10e
Sep 11 17:23:53.964176 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.964142    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"
Sep 11 17:23:53.964644 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.964186    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67d5b455cb-j422z" event={"ID":"d0452955-8e92-49a5-bae2-0a05e835ea9b","Type":"ContainerDied","Data":"e2b76f378bccc7bb219bc82dec50ad3759b48381c78de4eabe232e03d8126407"}
Sep 11 17:23:53.964644 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.964244    2569 scope.go:117] "RemoveContainer" containerID="4bc7e718a03f9f4a2c29adaec0c232427bc3bd138a705fef001412f44b5146ae"
Sep 11 17:23:53.966049 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.966010    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f" event={"ID":"a2660466-8270-4504-bc69-a8477d77782f","Type":"ContainerStarted","Data":"8ad1137c062c239f0ca93bb1eacf689f06629fab4a0e995ec129a59939685ddf"}
Sep 11 17:23:53.966179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.966069    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f" event={"ID":"a2660466-8270-4504-bc69-a8477d77782f","Type":"ContainerStarted","Data":"0d384c809ace445fe847debd9d3d7f79e78cd9e4e2fc9d8dc97c07566313b10e"}
Sep 11 17:23:53.966179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:53.966135    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:23:54.038367 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:54.038270    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f" podStartSLOduration=1.594364224 podStartE2EDuration="2.038254361s" totalImagesPullingTime="443.890137ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:23:54.009637935 +0000 UTC m=+2430.416856593" watchObservedRunningTime="2026-09-11 17:23:54.038254361 +0000 UTC m=+2430.445473083"
Sep 11 17:23:54.038703 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:54.038685    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"]
Sep 11 17:23:54.049573 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:54.049547    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-67d5b455cb-j422z"]
Sep 11 17:23:54.205132 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:54.205101    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0452955-8e92-49a5-bae2-0a05e835ea9b" path="/var/lib/kubelet/pods/d0452955-8e92-49a5-bae2-0a05e835ea9b/volumes"
Sep 11 17:23:59.379740 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:59.379702    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:23:59.977686 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:23:59.977656    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-bf6cf69f6-tws2f"
Sep 11 17:24:05.183189 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:05.183153    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:24:12.291756 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:12.291715    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-g5q5x"]
Sep 11 17:24:14.651174 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:14.651139    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:18.534908 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:18.534871    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:20.863660 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:20.863628    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-s58q5_0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5/manager/0.log"
Sep 11 17:24:21.278497 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:21.278461    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-r548q_45022cad-a452-41a1-a0cd-ed799faa9fac/server/0.log"
Sep 11 17:24:21.413863 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:21.413807    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-96pzb_20d779ce-f46a-4938-85e7-bbab7575ebf6/manager/0.log"
Sep 11 17:24:21.672324 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:21.672299    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d67cb4f96-qppv4_b8db771b-dba2-4a73-88f9-b2bfce087865/copy-manifests/0.log"
Sep 11 17:24:21.678473 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:21.678430    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d67cb4f96-qppv4_b8db771b-dba2-4a73-88f9-b2bfce087865/manager/0.log"
Sep 11 17:24:22.180464 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:22.180436    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-bf6cf69f6-tws2f_a2660466-8270-4504-bc69-a8477d77782f/maas-api/0.log"
Sep 11 17:24:22.542313 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:22.542237    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819115-q24tb_09623531-959d-4d48-b153-4accd0a14188/cleanup/0.log"
Sep 11 17:24:24.160523 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:24.160497    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-g5q5x_48573c38-144e-4ac3-8dd6-584af0fbc47f/limitador/0.log"
Sep 11 17:24:24.299409 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:24.299375    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-2rmt9_4ea846a3-9406-4e8b-a1e4-d178be88b5d5/manager/0.log"
Sep 11 17:24:24.680348 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:24.680311    2569 scope.go:117] "RemoveContainer" containerID="14f3f30bc27e2e854a81415370a3ea4455029abeae6034e17bc66855236bddd8"
Sep 11 17:24:24.689744 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:24.689707    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4_8a4fa0d6-9426-42f7-82b9-18ef74086496/istio-proxy/0.log"
Sep 11 17:24:25.194356 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:25.194299    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fq525_1006fc20-15e3-4325-9e4b-bc09745e2672/istio-proxy/1.log"
Sep 11 17:24:25.236363 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:25.236327    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:25.778713 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:25.778676    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:25.840773 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:25.840729    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:26.315456 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:26.315419    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-whg6n_8bffcfb2-7c5b-4921-9f1f-c76a7b3a501a/main/0.log"
Sep 11 17:24:26.573406 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:26.573317    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-2cwf9_17ffca60-3170-4fad-a29f-235be9ea7ae1/main/0.log"
Sep 11 17:24:32.103668 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.103630    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4wcq9/must-gather-5fkz5"]
Sep 11 17:24:32.104058 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.104012    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0452955-8e92-49a5-bae2-0a05e835ea9b" containerName="maas-api"
Sep 11 17:24:32.104058 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.104023    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0452955-8e92-49a5-bae2-0a05e835ea9b" containerName="maas-api"
Sep 11 17:24:32.104157 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.104113    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0452955-8e92-49a5-bae2-0a05e835ea9b" containerName="maas-api"
Sep 11 17:24:32.107271 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.107253    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wcq9/must-gather-5fkz5"
Sep 11 17:24:32.109559 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.109525    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4wcq9\"/\"kube-root-ca.crt\""
Sep 11 17:24:32.109682 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.109598    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4wcq9\"/\"default-dockercfg-g5zvq\""
Sep 11 17:24:32.110025 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.110006    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4wcq9\"/\"openshift-service-ca.crt\""
Sep 11 17:24:32.116530 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.116501    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:32.117092 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.117051    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wcq9/must-gather-5fkz5"]
Sep 11 17:24:32.182067 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.182033    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:32.285979 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.285938    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4616dc26-7d24-4e4b-8ec4-73882f08783f-must-gather-output\") pod \"must-gather-5fkz5\" (UID: \"4616dc26-7d24-4e4b-8ec4-73882f08783f\") " pod="openshift-must-gather-4wcq9/must-gather-5fkz5"
Sep 11 17:24:32.286209 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.286051    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqxkx\" (UniqueName: \"kubernetes.io/projected/4616dc26-7d24-4e4b-8ec4-73882f08783f-kube-api-access-mqxkx\") pod \"must-gather-5fkz5\" (UID: \"4616dc26-7d24-4e4b-8ec4-73882f08783f\") " pod="openshift-must-gather-4wcq9/must-gather-5fkz5"
Sep 11 17:24:32.387411 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.387384    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4616dc26-7d24-4e4b-8ec4-73882f08783f-must-gather-output\") pod \"must-gather-5fkz5\" (UID: \"4616dc26-7d24-4e4b-8ec4-73882f08783f\") " pod="openshift-must-gather-4wcq9/must-gather-5fkz5"
Sep 11 17:24:32.387618 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.387484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqxkx\" (UniqueName: \"kubernetes.io/projected/4616dc26-7d24-4e4b-8ec4-73882f08783f-kube-api-access-mqxkx\") pod \"must-gather-5fkz5\" (UID: \"4616dc26-7d24-4e4b-8ec4-73882f08783f\") " pod="openshift-must-gather-4wcq9/must-gather-5fkz5"
Sep 11 17:24:32.387771 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.387749    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4616dc26-7d24-4e4b-8ec4-73882f08783f-must-gather-output\") pod \"must-gather-5fkz5\" (UID: \"4616dc26-7d24-4e4b-8ec4-73882f08783f\") " pod="openshift-must-gather-4wcq9/must-gather-5fkz5"
Sep 11 17:24:32.396179 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.396162    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqxkx\" (UniqueName: \"kubernetes.io/projected/4616dc26-7d24-4e4b-8ec4-73882f08783f-kube-api-access-mqxkx\") pod \"must-gather-5fkz5\" (UID: \"4616dc26-7d24-4e4b-8ec4-73882f08783f\") " pod="openshift-must-gather-4wcq9/must-gather-5fkz5"
Sep 11 17:24:32.417243 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.417220    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wcq9/must-gather-5fkz5"
Sep 11 17:24:32.688737 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:32.688709    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wcq9/must-gather-5fkz5"]
Sep 11 17:24:32.691715 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:24:32.691681    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4616dc26_7d24_4e4b_8ec4_73882f08783f.slice/crio-beaf6bc449d0958cf42b9b6c7ec6922769d551d5f7868fe5a3f20b072f3d1bc0 WatchSource:0}: Error finding container beaf6bc449d0958cf42b9b6c7ec6922769d551d5f7868fe5a3f20b072f3d1bc0: Status 404 returned error can't find the container with id beaf6bc449d0958cf42b9b6c7ec6922769d551d5f7868fe5a3f20b072f3d1bc0
Sep 11 17:24:33.167739 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:33.167703    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wcq9/must-gather-5fkz5" event={"ID":"4616dc26-7d24-4e4b-8ec4-73882f08783f","Type":"ContainerStarted","Data":"beaf6bc449d0958cf42b9b6c7ec6922769d551d5f7868fe5a3f20b072f3d1bc0"}
Sep 11 17:24:34.177772 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:34.177671    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wcq9/must-gather-5fkz5" event={"ID":"4616dc26-7d24-4e4b-8ec4-73882f08783f","Type":"ContainerStarted","Data":"22451a4b331c97e12fec931ce3384aebe20e8551ddca4eb476418ea272d98de3"}
Sep 11 17:24:34.177772 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:34.177730    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wcq9/must-gather-5fkz5" event={"ID":"4616dc26-7d24-4e4b-8ec4-73882f08783f","Type":"ContainerStarted","Data":"e98838847478bb7bb427f24fe640820f38907c066ce382ca0b61309b597e1051"}
Sep 11 17:24:34.195611 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:34.195545    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4wcq9/must-gather-5fkz5" podStartSLOduration=1.458466828 podStartE2EDuration="2.19552148s" totalImagesPullingTime="737.054652ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:24:34.193870754 +0000 UTC m=+2470.601089403" watchObservedRunningTime="2026-09-11 17:24:34.19552148 +0000 UTC m=+2470.602740130"
Sep 11 17:24:35.179424 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:35.179388    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b6fm6_a5e339d2-e61e-4cf6-8f35-aafb644b4136/global-pull-secret-syncer/0.log"
Sep 11 17:24:35.312491 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:35.312457    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jpzqw_f26a8019-80a6-4f41-b39c-8417143279b8/konnectivity-agent/0.log"
Sep 11 17:24:35.397649 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:35.397612    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-55.ec2.internal_93e5adb587d6c9f5d90710b23c25b76f/haproxy/0.log"
Sep 11 17:24:39.806852 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:39.806826    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-g5q5x_48573c38-144e-4ac3-8dd6-584af0fbc47f/limitador/0.log"
Sep 11 17:24:39.889697 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:39.889668    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-2rmt9_4ea846a3-9406-4e8b-a1e4-d178be88b5d5/manager/0.log"
Sep 11 17:24:41.715098 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:41.715045    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-wxsp7_bc0b8a1a-a005-4d8d-a407-a93598edfc6d/monitoring-plugin/0.log"
Sep 11 17:24:41.744110 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:41.744059    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8thxd_1189ee6b-cf59-424e-82bc-e1b7d8345eaf/node-exporter/0.log"
Sep 11 17:24:41.767981 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:41.767944    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8thxd_1189ee6b-cf59-424e-82bc-e1b7d8345eaf/kube-rbac-proxy/0.log"
Sep 11 17:24:41.788044 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:41.787952    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8thxd_1189ee6b-cf59-424e-82bc-e1b7d8345eaf/init-textfile/0.log"
Sep 11 17:24:42.058395 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.058303    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-2wnhh_9e1dfad2-13dd-4480-b68b-4a711edf9c29/kube-rbac-proxy-main/0.log"
Sep 11 17:24:42.087116 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.087067    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-2wnhh_9e1dfad2-13dd-4480-b68b-4a711edf9c29/kube-rbac-proxy-self/0.log"
Sep 11 17:24:42.111913 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.111863    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-2wnhh_9e1dfad2-13dd-4480-b68b-4a711edf9c29/openshift-state-metrics/0.log"
Sep 11 17:24:42.164019 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.163929    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71105af7-a093-4ea9-8711-dd717ec483f8/prometheus/0.log"
Sep 11 17:24:42.184582 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.184551    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71105af7-a093-4ea9-8711-dd717ec483f8/config-reloader/0.log"
Sep 11 17:24:42.221538 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.221465    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71105af7-a093-4ea9-8711-dd717ec483f8/thanos-sidecar/0.log"
Sep 11 17:24:42.246610 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.246586    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71105af7-a093-4ea9-8711-dd717ec483f8/kube-rbac-proxy-web/0.log"
Sep 11 17:24:42.271096 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.271057    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71105af7-a093-4ea9-8711-dd717ec483f8/kube-rbac-proxy/0.log"
Sep 11 17:24:42.299017 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.298923    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71105af7-a093-4ea9-8711-dd717ec483f8/kube-rbac-proxy-thanos/0.log"
Sep 11 17:24:42.325951 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.325859    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71105af7-a093-4ea9-8711-dd717ec483f8/init-config-reloader/0.log"
Sep 11 17:24:42.364495 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.364463    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-x8lmv_3571c72c-acb0-4605-836a-3f8711ee488e/prometheus-operator/0.log"
Sep 11 17:24:42.398099 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.396838    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-x8lmv_3571c72c-acb0-4605-836a-3f8711ee488e/kube-rbac-proxy/0.log"
Sep 11 17:24:42.428134 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.428101    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-ngkps_14501227-b8bc-4233-9fd6-376798609c08/prometheus-operator-admission-webhook/0.log"
Sep 11 17:24:42.552643 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.552563    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7ff974997c-trpkx_fd73d7c4-ea76-4820-8c34-c74f014fa722/thanos-query/0.log"
Sep 11 17:24:42.586017 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.585912    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7ff974997c-trpkx_fd73d7c4-ea76-4820-8c34-c74f014fa722/kube-rbac-proxy-web/0.log"
Sep 11 17:24:42.606821 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.606796    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7ff974997c-trpkx_fd73d7c4-ea76-4820-8c34-c74f014fa722/kube-rbac-proxy/0.log"
Sep 11 17:24:42.627694 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.627662    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7ff974997c-trpkx_fd73d7c4-ea76-4820-8c34-c74f014fa722/prom-label-proxy/0.log"
Sep 11 17:24:42.647731 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.647695    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7ff974997c-trpkx_fd73d7c4-ea76-4820-8c34-c74f014fa722/kube-rbac-proxy-rules/0.log"
Sep 11 17:24:42.669690 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:42.669658    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7ff974997c-trpkx_fd73d7c4-ea76-4820-8c34-c74f014fa722/kube-rbac-proxy-metrics/0.log"
Sep 11 17:24:44.199100 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:44.199058    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-6cmcv_28b211cd-7cd1-4ff3-9280-3001eae82504/networking-console-plugin/0.log"
Sep 11 17:24:45.472290 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:45.472244    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-gvcnc_2ca0adb8-a1e2-42fe-b506-67c40d3de813/download-server/0.log"
Sep 11 17:24:45.825851 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:45.825763    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"]
Sep 11 17:24:45.832747 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:45.832714    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:45.837916 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:45.837286    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"]
Sep 11 17:24:45.937344 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:45.937303    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-lib-modules\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:45.937641 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:45.937617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp4zc\" (UniqueName: \"kubernetes.io/projected/3e9166f6-3c69-460e-979d-d727b5b87f48-kube-api-access-hp4zc\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:45.937808 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:45.937790    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-podres\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:45.937998 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:45.937981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-sys\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:45.938167 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:45.938151    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-proc\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.039630 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.039583    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-sys\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.039820 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.039645    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-proc\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.039820 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.039713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-lib-modules\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.039820 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.039764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp4zc\" (UniqueName: \"kubernetes.io/projected/3e9166f6-3c69-460e-979d-d727b5b87f48-kube-api-access-hp4zc\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.039820 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.039801    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-podres\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.040035 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.040013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-podres\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.040124 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.040106    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-sys\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.040194 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.040170    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-proc\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.040292 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.040272    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e9166f6-3c69-460e-979d-d727b5b87f48-lib-modules\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.048549 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.048526    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp4zc\" (UniqueName: \"kubernetes.io/projected/3e9166f6-3c69-460e-979d-d727b5b87f48-kube-api-access-hp4zc\") pod \"perf-node-gather-daemonset-w6d64\" (UID: \"3e9166f6-3c69-460e-979d-d727b5b87f48\") " pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.150910 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.150869    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:46.445905 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:46.445735    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"]
Sep 11 17:24:46.451206 ip-10-0-135-55 kubenswrapper[2569]: W0911 17:24:46.451157    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3e9166f6_3c69_460e_979d_d727b5b87f48.slice/crio-2e5e90fd6ddd2dbd14bb2c7c2a85b1eab3776c8f450d18eb65f14fdb89221dcf WatchSource:0}: Error finding container 2e5e90fd6ddd2dbd14bb2c7c2a85b1eab3776c8f450d18eb65f14fdb89221dcf: Status 404 returned error can't find the container with id 2e5e90fd6ddd2dbd14bb2c7c2a85b1eab3776c8f450d18eb65f14fdb89221dcf
Sep 11 17:24:47.284844 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:47.284805    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64" event={"ID":"3e9166f6-3c69-460e-979d-d727b5b87f48","Type":"ContainerStarted","Data":"1693e20ae66855e2df519f63b135f2fe74f4a94fa1b5f348822db9e5db6357ba"}
Sep 11 17:24:47.285301 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:47.284868    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64" event={"ID":"3e9166f6-3c69-460e-979d-d727b5b87f48","Type":"ContainerStarted","Data":"2e5e90fd6ddd2dbd14bb2c7c2a85b1eab3776c8f450d18eb65f14fdb89221dcf"}
Sep 11 17:24:47.285301 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:47.284928    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:47.305604 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:47.305529    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64" podStartSLOduration=2.3055032 podStartE2EDuration="2.3055032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:24:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:24:47.302990292 +0000 UTC m=+2483.710208961" watchObservedRunningTime="2026-09-11 17:24:47.3055032 +0000 UTC m=+2483.712721849"
Sep 11 17:24:47.557754 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:47.557668    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wpfmb_dc500b91-6880-4405-92d9-21e5cc455012/dns/0.log"
Sep 11 17:24:47.577766 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:47.577734    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wpfmb_dc500b91-6880-4405-92d9-21e5cc455012/kube-rbac-proxy/0.log"
Sep 11 17:24:47.671303 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:47.671274    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gd8hl_582ece86-42d2-4803-853b-611a440d509b/dns-node-resolver/0.log"
Sep 11 17:24:48.283357 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:48.283227    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8f7d7fc8c-4pwmv_e21fe60e-04d0-45fd-8d85-0bb411828a0f/registry/0.log"
Sep 11 17:24:48.332904 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:48.332873    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l28sb_7fa5fa96-b9a1-4392-8189-3944be73baba/node-ca/0.log"
Sep 11 17:24:49.412424 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:49.412359    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4_8a4fa0d6-9426-42f7-82b9-18ef74086496/istio-proxy/0.log"
Sep 11 17:24:49.917791 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:49.917755    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fq525_1006fc20-15e3-4325-9e4b-bc09745e2672/istio-proxy/0.log"
Sep 11 17:24:49.923517 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:49.923455    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fq525_1006fc20-15e3-4325-9e4b-bc09745e2672/istio-proxy/1.log"
Sep 11 17:24:50.917127 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:50.917091    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mtrfv_e42d329c-4c8d-442c-bf85-0a9c6f2d49e3/serve-healthcheck-canary/0.log"
Sep 11 17:24:51.510708 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:51.510677    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d82bq_64f3864a-ce43-4330-b5cc-c43bd0f045e3/kube-rbac-proxy/0.log"
Sep 11 17:24:51.529493 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:51.529461    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d82bq_64f3864a-ce43-4330-b5cc-c43bd0f045e3/exporter/0.log"
Sep 11 17:24:51.549122 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:51.549094    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d82bq_64f3864a-ce43-4330-b5cc-c43bd0f045e3/extractor/0.log"
Sep 11 17:24:53.304189 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:53.304153    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4wcq9/perf-node-gather-daemonset-w6d64"
Sep 11 17:24:53.817227 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:53.817184    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-s58q5_0b4dd94c-94bd-4fb6-83ef-ec0f2cb725e5/manager/0.log"
Sep 11 17:24:54.133148 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:54.133123    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-r548q_45022cad-a452-41a1-a0cd-ed799faa9fac/server/0.log"
Sep 11 17:24:54.282566 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:54.282532    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-96pzb_20d779ce-f46a-4938-85e7-bbab7575ebf6/manager/0.log"
Sep 11 17:24:54.413368 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:54.413293    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d67cb4f96-qppv4_b8db771b-dba2-4a73-88f9-b2bfce087865/manager/0.log"
Sep 11 17:24:54.458402 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:54.458377    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d67cb4f96-qppv4_b8db771b-dba2-4a73-88f9-b2bfce087865/copy-manifests/0.log"
Sep 11 17:24:56.070447 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:56.070414    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-67c876df9b-ghrtp_648e3cb0-db9c-42db-b080-9f4b0d4f4d5f/manager/0.log"
Sep 11 17:24:56.124391 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:56.124363    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-cv5g6_c792a287-5a78-4c7b-a6db-a4893b658fc0/openshift-lws-operator/0.log"
Sep 11 17:24:56.241029 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:56.240940    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4_8a4fa0d6-9426-42f7-82b9-18ef74086496/istio-proxy/0.log"
Sep 11 17:24:56.717015 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:56.716968    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fq525_1006fc20-15e3-4325-9e4b-bc09745e2672/istio-proxy/0.log"
Sep 11 17:24:56.722730 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:56.722695    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fq525_1006fc20-15e3-4325-9e4b-bc09745e2672/istio-proxy/1.log"
Sep 11 17:24:58.295743 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:58.295663    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qt6q4_8a4fa0d6-9426-42f7-82b9-18ef74086496/istio-proxy/0.log"
Sep 11 17:24:58.804803 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:58.804771    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fq525_1006fc20-15e3-4325-9e4b-bc09745e2672/istio-proxy/0.log"
Sep 11 17:24:58.810012 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:24:58.809982    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fq525_1006fc20-15e3-4325-9e4b-bc09745e2672/istio-proxy/1.log"
Sep 11 17:25:02.818934 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:25:02.818907    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6ngsr_4604c98e-255c-4fa1-9266-08827d0e3761/kube-multus-additional-cni-plugins/0.log"
Sep 11 17:25:02.840009 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:25:02.839962    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6ngsr_4604c98e-255c-4fa1-9266-08827d0e3761/egress-router-binary-copy/0.log"
Sep 11 17:25:02.859882 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:25:02.859853    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6ngsr_4604c98e-255c-4fa1-9266-08827d0e3761/cni-plugins/0.log"
Sep 11 17:25:02.880523 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:25:02.880499    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6ngsr_4604c98e-255c-4fa1-9266-08827d0e3761/bond-cni-plugin/0.log"
Sep 11 17:25:02.902458 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:25:02.902426    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6ngsr_4604c98e-255c-4fa1-9266-08827d0e3761/routeoverride-cni/0.log"
Sep 11 17:25:02.924242 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:25:02.924207    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6ngsr_4604c98e-255c-4fa1-9266-08827d0e3761/whereabouts-cni-bincopy/0.log"
Sep 11 17:25:02.945980 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:25:02.945946    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6ngsr_4604c98e-255c-4fa1-9266-08827d0e3761/whereabouts-cni/0.log"
Sep 11 17:25:03.379518 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:25:03.379483    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vtmr5_18ea8e25-6023-41a7-8643-9f093ffeab1e/kube-multus/0.log"
Sep 11 17:25:03.546583 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:25:03.546551    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xnn4g_ef8b79f7-feb0-46de-898c-1db3ebd142db/network-metrics-daemon/0.log"
Sep 11 17:25:03.563561 ip-10-0-135-55 kubenswrapper[2569]: I0911 17:25:03.563534    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xnn4g_ef8b79f7-feb0-46de-898c-1db3ebd142db/kube-rbac-proxy/0.log"