Sep 08 14:14:27.407255 ip-10-0-131-22 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 08 14:14:27.407266 ip-10-0-131-22 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 08 14:14:27.407273 ip-10-0-131-22 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 08 14:14:27.407505 ip-10-0-131-22 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 08 14:14:37.444306 ip-10-0-131-22 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 08 14:14:37.444324 ip-10-0-131-22 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 41914dfa122043da9569932a58102024 --
Sep 08 14:16:24.658752 ip-10-0-131-22 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 14:16:25.085488 ip-10-0-131-22 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 14:16:25.085488 ip-10-0-131-22 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 14:16:25.085488 ip-10-0-131-22 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 14:16:25.085488 ip-10-0-131-22 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 14:16:25.085488 ip-10-0-131-22 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 14:16:25.086140 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.085530    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 14:16:25.089722 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089573    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:16:25.089722 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089722    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089727    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089730    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089733    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089737    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089740    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089745    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089749    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089752    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089756    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089760    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089763    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089766    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089777    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089780    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089785    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089787    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:16:25.089789 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089796    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089798    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089802    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089805    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089808    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089811    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089814    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089816    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089819    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089822    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089824    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089827    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089830    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089834    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089837    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089840    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089842    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089845    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089847    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:16:25.090190 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089850    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089853    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089855    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089858    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089860    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089863    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089866    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089868    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089871    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089873    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089877    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089879    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089882    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089884    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089887    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089891    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089894    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089896    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089899    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089901    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:16:25.090697 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089904    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089907    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089909    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089912    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089914    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089917    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089920    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089922    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089925    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089927    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089929    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089932    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089936    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089938    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089941    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089943    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089946    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089948    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089951    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089954    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:16:25.091177 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089956    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089959    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089962    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089971    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089973    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089976    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089979    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089982    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089984    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089987    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089990    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089994    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089996    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.089999    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090002    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090004    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090007    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090022    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090025    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090027    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:16:25.091668 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090030    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090033    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090761    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090768    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090771    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090774    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090777    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090780    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090783    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090785    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090788    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090791    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090794    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090796    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090799    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090802    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090805    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090808    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090811    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090814    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:16:25.092138 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090816    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090820    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090824    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090827    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090830    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090832    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090835    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090839    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090843    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090846    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090849    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090852    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090854    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090857    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090859    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090862    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090864    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090867    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:16:25.092642 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090869    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090872    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090875    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090877    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090880    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090883    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090887    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090891    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090894    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090897    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090899    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090903    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090905    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090908    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090911    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090913    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090916    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090919    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090921    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:16:25.093088 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090924    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090926    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090929    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090931    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090934    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090936    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090939    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090941    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090944    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090946    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090949    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090952    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090955    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090957    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090960    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090963    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090965    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090969    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090971    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090974    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:16:25.093568 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090977    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090980    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090983    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090986    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090988    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090991    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090994    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090997    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.090999    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091002    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091004    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091007    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091009    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091012    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091014    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091017    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091019    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091022    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091024    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091027    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:16:25.094052 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091030    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091032    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091034    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091037    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091442    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091449    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091452    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091455    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091458    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091461    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091464    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091467    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091470    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091473    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091476    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091479    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091482    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091485    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091488    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091491    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:16:25.094578 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091493    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091496    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091500    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091503    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091507    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091509    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091512    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091515    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091519    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091522    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091524    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091527    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091530    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091532    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091535    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091537    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091542    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091545    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091547    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:16:25.095061 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091550    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091552    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091556    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091560    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091564    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091566    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091569    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091572    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091576    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091578    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091581    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091584    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091587    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091590    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091593    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091595    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091597    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091600    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091602    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:16:25.095546 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091605    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091608    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091612    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091615    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091617    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091620    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091622    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091625    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091627    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091630    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091634    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091636    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091639    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091642    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091645    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091647    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091650    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091652    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091655    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091657    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:16:25.096044 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091660    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091663    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091665    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091667    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091670    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091673    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091676    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091679    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091681    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091684    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091686    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091689    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091691    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091694    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091697    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091700    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091702    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091705    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091707    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091710    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:16:25.096542 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091712    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091715    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091719    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091722    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.091724    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091789    2558 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091822    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091828    2558 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091832    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091836    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091840    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091843    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091847    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091850    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091853    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091857    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091860    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091863    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091867    2558 flags.go:64] FLAG: --cgroup-root=""
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091870    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091873    2558 flags.go:64] FLAG: --client-ca-file=""
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091876    2558 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091879    2558 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091883    2558 flags.go:64] FLAG: --cluster-domain=""
Sep 08 14:16:25.097026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091886    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091889    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091897    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091900    2558 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091904    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091907    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091910    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091913    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091916    2558 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091919    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091923    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091926    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091929    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091933    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091936    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091939    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091941    2558 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091944    2558 flags.go:64] FLAG: --enable-server="true"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091947    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091952    2558 flags.go:64] FLAG: --event-burst="100"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091955    2558 flags.go:64] FLAG: --event-qps="50"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091958    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091960    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091963    2558 flags.go:64] FLAG: --eviction-hard=""
Sep 08 14:16:25.097641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091967    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091969    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091973    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091976    2558 flags.go:64] FLAG: --eviction-soft=""
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091979    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091982    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091985    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091987    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091990    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091993    2558 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.091997    2558 flags.go:64] FLAG: --feature-gates=""
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092001    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092004    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092007    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092009    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092012    2558 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092016    2558 flags.go:64] FLAG: --help="false"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092018    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092023    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092026    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092029    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092032    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092036    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092039    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 14:16:25.098222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092041    2558 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092044    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092047    2558 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092050    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092053    2558 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092055    2558 flags.go:64] FLAG: --kube-reserved=""
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092058    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092061    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092064    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092067    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092070    2558 flags.go:64] FLAG: --lock-file=""
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092072    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092076    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092079    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092084    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092087    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092090    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092092    2558 flags.go:64] FLAG: --logging-format="text"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092096    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092099    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092102    2558 flags.go:64] FLAG: --manifest-url=""
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092105    2558 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092109    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092112    2558 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092115    2558 flags.go:64] FLAG: --max-pods="110"
Sep 08 14:16:25.098830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092118    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092123    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092126    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092129    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092135    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092138    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092142    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092149    2558 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092152    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092155    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092158    2558 flags.go:64] FLAG: --pod-cidr=""
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092161    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092166    2558 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092169    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092172    2558 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092175    2558 flags.go:64] FLAG: --port="10250"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092178    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092181    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00c8e79bca59993ab"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092184    2558 flags.go:64] FLAG: --qos-reserved=""
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092188    2558 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092191    2558 flags.go:64] FLAG: --register-node="true"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092194    2558 flags.go:64] FLAG: --register-with-taints=""
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092198    2558 flags.go:64] FLAG: --registry-burst="10"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092214    2558 flags.go:64] FLAG: --registry-qps="5"
Sep 08 14:16:25.099445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092217    2558 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092221    2558 flags.go:64] FLAG: --reserved-memory=""
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092224    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092228    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092231    2558 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092233    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092236    2558 flags.go:64] FLAG: --runonce="false"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092239    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092242    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092245    2558 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092248    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092251    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092254    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092257    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092260    2558 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092263    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092266    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092269    2558 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092271    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092274    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092277    2558 flags.go:64] FLAG: --system-cgroups=""
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092280    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092285    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092288    2558 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092290    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 14:16:25.100018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092295    2558 flags.go:64] FLAG: --tls-min-version=""
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092298    2558 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092301    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092304    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092307    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092310    2558 flags.go:64] FLAG: --v="2"
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092314    2558 flags.go:64] FLAG: --version="false"
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092318    2558 flags.go:64] FLAG: --vmodule=""
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092321    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.092325    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092438    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092442    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092445    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092447    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092450    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092452    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092455    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092458    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092460    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092463    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092465    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092468    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:16:25.100706 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092470    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092473    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092476    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092478    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092481    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092483    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092486    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092488    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092491    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092494    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092496    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092499    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092502    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092505    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092507    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092510    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092513    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092515    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092518    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092520    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:16:25.101545 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092523    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092525    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092528    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092530    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092533    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092535    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092538    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092541    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092543    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092547    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092551    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092553    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092556    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092559    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092561    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092563    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092566    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092569    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092572    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:16:25.102039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092576    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092579    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092582    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092585    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092588    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092591    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092593    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092596    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092599    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092602    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092604    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092607    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092610    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092613    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092615    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092618    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092621    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092624    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092626    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092629    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:16:25.102516 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092631    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092635    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092637    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092641    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092645    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092648    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092650    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092653    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092656    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092659    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092662    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092664    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092667    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092670    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092674    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092677    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092679    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092681    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092684    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:16:25.103101 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092687    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:16:25.103585 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092689    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:16:25.103585 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092692    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:16:25.103585 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092696    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:16:25.103585 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092700    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:16:25.103585 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092704    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:16:25.103585 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092708    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:16:25.103585 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092712    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:16:25.103585 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.092714    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:16:25.103585 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.093457    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 14:16:25.103810 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.103722    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 14:16:25.103810 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.103737    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 14:16:25.103810 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.103752    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 14:16:25.103810 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.103757    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 14:16:25.103810 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103812    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103818    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103821    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103824    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103829    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103833    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103836    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103839    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103842    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103845    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103847    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103850    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103853    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103855    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103858    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103861    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103864    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103866    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103869    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:16:25.103944 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103871    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103878    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103881    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103884    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103887    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103890    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103892    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103895    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103898    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103901    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103903    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103906    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103909    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103912    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103915    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103917    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103920    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103923    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103926    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103929    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:16:25.104428 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103931    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103934    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103936    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103939    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103942    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103944    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103947    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103950    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103953    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103956    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103958    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103961    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103964    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103966    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103968    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103971    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103973    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103976    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103979    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103982    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:16:25.104915 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103984    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103987    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103991    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103994    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.103997    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104001    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104004    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104007    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104009    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104013    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104016    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104019    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104021    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104025    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104028    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104030    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104033    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104036    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104039    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:16:25.105507 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104041    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104045    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104048    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104051    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104053    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104056    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104059    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104061    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104064    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104066    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104069    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104072    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104075    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104077    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104080    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104082    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104085    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104087    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104090    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:16:25.105962 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104093    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104095    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.104100    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104236    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104242    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104245    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104248    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104250    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104253    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104255    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104258    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104260    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104264    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104268    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:16:25.106443 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104271    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104275    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104279    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104282    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104285    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104288    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104290    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104293    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104295    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104298    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104301    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104303    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104306    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104308    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104311    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104314    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104316    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104318    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104321    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:16:25.106785 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104324    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104326    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104329    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104331    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104334    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104337    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104339    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104342    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104344    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104348    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104351    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104354    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104356    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104359    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104361    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104364    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104367    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104370    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104373    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104375    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:16:25.107259 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104378    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104381    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104383    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104386    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104388    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104391    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104393    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104396    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104398    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104401    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104403    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104406    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104408    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104411    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104413    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104416    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104418    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104420    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104423    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104425    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:16:25.107739 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104428    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104430    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104433    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104436    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104438    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104441    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104444    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104447    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104450    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104453    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104456    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104458    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104461    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104464    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104466    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104469    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104471    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104474    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104476    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104479    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:16:25.108227 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104481    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:16:25.108726 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104484    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:16:25.108726 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104486    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:16:25.108726 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104489    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:16:25.108726 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104491    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:16:25.108726 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104494    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:16:25.108726 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104496    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:16:25.108726 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104498    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:16:25.108726 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:25.104501    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:16:25.108726 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.104506    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 14:16:25.108726 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.104606    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 14:16:25.109290 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.109277    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 14:16:25.110301 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.110267    2558 server.go:1019] "Starting client certificate rotation"
Sep 08 14:16:25.110406 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.110373    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 14:16:25.110406 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.110406    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 14:16:25.112975 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.112963    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 14:16:25.114072 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.114055    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 14:16:25.132873 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.132850    2558 log.go:25] "Validated CRI v1 runtime API"
Sep 08 14:16:25.136705 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.136692    2558 log.go:25] "Validated CRI v1 image API"
Sep 08 14:16:25.137659 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.137640    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 14:16:25.141130 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.141112    2558 fs.go:135] Filesystem UUIDs: map[00aefb45-66d9-4202-b3d6-889876dd16dc:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a9db2a16-7f1e-4591-aa73-b583e9b76772:/dev/nvme0n1p3]
Sep 08 14:16:25.141188 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.141129    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 14:16:25.147910 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.147795    2558 manager.go:216] Machine: {Timestamp:2026-09-08 14:16:25.146450377 +0000 UTC m=+0.377381567 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095186 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20c18cb616a2b8080fc483473de5ca SystemUUID:ec20c18c-b616-a2b8-080f-c483473de5ca BootID:41914dfa-1220-43da-9569-932a58102024 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2c:b6:05:4f:ed Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2c:b6:05:4f:ed Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:24:0b:cf:a9:ef Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 14:16:25.147910 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.147898    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 14:16:25.148028 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.148008    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 14:16:25.149025 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.149009    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 14:16:25.151696 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.151670    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 14:16:25.151829 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.151700    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-22.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 14:16:25.152521 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.152512    2558 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 14:16:25.152557 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.152523    2558 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 14:16:25.152557 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.152530    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 14:16:25.153276 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.153265    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 14:16:25.155103 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.155093    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 14:16:25.159384 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.159373    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 14:16:25.159494 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.159485    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 14:16:25.159536 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.159515    2558 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 14:16:25.159536 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.159524    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 14:16:25.159600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.159538    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 14:16:25.159600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.159547    2558 kubelet.go:398] "Adding apiserver pod source"
Sep 08 14:16:25.159600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.159557    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 14:16:25.159930 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.159917    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 14:16:25.159962 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.159943    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 14:16:25.162964 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.162947    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 14:16:25.165789 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.165774    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 14:16:25.165861 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.165824    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 14:16:25.166716 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166702    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 14:16:25.166766 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166723    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 14:16:25.166766 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166731    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 14:16:25.166766 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166737    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 14:16:25.166766 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166746    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 14:16:25.166766 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166752    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 14:16:25.166766 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166766    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 14:16:25.166943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166771    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 14:16:25.166943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166778    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 14:16:25.166943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166783    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 14:16:25.166943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166792    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 14:16:25.166943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.166797    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 14:16:25.168475 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.168458    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 14:16:25.168475 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.168477    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 14:16:25.170655 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.170628    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-22.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 14:16:25.170737 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.170630    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 14:16:25.172689 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.172673    2558 server.go:1268] "Started kubelet"
Sep 08 14:16:25.172765 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.172734    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 14:16:25.172844 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.172805    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 14:16:25.172893 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.172861    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 14:16:25.173328 ip-10-0-131-22 systemd[1]: Started Kubernetes Kubelet.
Sep 08 14:16:25.174431 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.174402    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 14:16:25.175533 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.175517    2558 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 14:16:25.177899 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.177882    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 14:16:25.180120 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.180102    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 14:16:25.180120 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.180114    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 14:16:25.180699 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.180659    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:25.180936 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.180914    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 14:16:25.181010 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.180958    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 14:16:25.181091 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.180916    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 14:16:25.181239 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.181109    2558 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 14:16:25.181239 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.181126    2558 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 14:16:25.182515 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.182494    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 14:16:25.182515 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.182514    2558 factory.go:55] Registering systemd factory
Sep 08 14:16:25.182639 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.182522    2558 factory.go:223] Registration of the systemd container factory successfully
Sep 08 14:16:25.182762 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.182739    2558 factory.go:153] Registering CRI-O factory
Sep 08 14:16:25.182762 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.182750    2558 factory.go:223] Registration of the crio container factory successfully
Sep 08 14:16:25.182831 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.182766    2558 factory.go:103] Registering Raw factory
Sep 08 14:16:25.182831 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.182779    2558 manager.go:1196] Started watching for new ooms in manager
Sep 08 14:16:25.183524 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.183505    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-22.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 08 14:16:25.183524 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.183518    2558 manager.go:349] Starting recovery of all containers
Sep 08 14:16:25.183642 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.183604    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 08 14:16:25.183713 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.183697    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-22.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 14:16:25.187271 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.183604    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-22.ec2.internal.18d35e23e016384e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-22.ec2.internal,UID:ip-10-0-131-22.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-22.ec2.internal,},FirstTimestamp:2026-09-08 14:16:25.172637774 +0000 UTC m=+0.403568962,LastTimestamp:2026-09-08 14:16:25.172637774 +0000 UTC m=+0.403568962,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-22.ec2.internal,}"
Sep 08 14:16:25.198666 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.198645    2558 manager.go:354] Recovery completed
Sep 08 14:16:25.202435 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.202411    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xcpqp"
Sep 08 14:16:25.204605 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.204593    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:16:25.206521 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.206504    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:16:25.206605 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.206534    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:16:25.206605 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.206547    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:16:25.207164 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.207148    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 14:16:25.207164 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.207161    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 14:16:25.207164 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.207164    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:16:25.207382 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.207177    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 14:16:25.207871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.207858    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:16:25.207923 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.207883    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:16:25.207923 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.207895    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:16:25.208423 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.208358    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-22.ec2.internal.18d35e23e21b36de  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-22.ec2.internal,UID:ip-10-0-131-22.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-22.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-22.ec2.internal,},FirstTimestamp:2026-09-08 14:16:25.206519518 +0000 UTC m=+0.437450706,LastTimestamp:2026-09-08 14:16:25.206519518 +0000 UTC m=+0.437450706,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-22.ec2.internal,}"
Sep 08 14:16:25.209298 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.209285    2558 policy_none.go:49] "None policy: Start"
Sep 08 14:16:25.209358 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.209302    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 14:16:25.209358 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.209312    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 14:16:25.211287 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.211273    2558 policy_none.go:47] "Start"
Sep 08 14:16:25.211347 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.211303    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xcpqp"
Sep 08 14:16:25.258715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.256788    2558 manager.go:337] "Starting Device Plugin manager"
Sep 08 14:16:25.258715 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.256837    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 14:16:25.258715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.256853    2558 server.go:87] "Starting device plugin registration server"
Sep 08 14:16:25.258715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.257129    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 14:16:25.258715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.257140    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 14:16:25.258715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.257227    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 14:16:25.258715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.257315    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 14:16:25.258715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.257324    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 14:16:25.258715 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.257902    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 14:16:25.258715 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.257954    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:25.279093 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.279066    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 14:16:25.280244 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.280224    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 14:16:25.280324 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.280265    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 14:16:25.280324 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.280287    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 14:16:25.280425 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.280327    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 08 14:16:25.282339 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.282320    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:16:25.358255 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.358100    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:16:25.359064 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.359046    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:16:25.359111 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.359082    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:16:25.359111 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.359097    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:16:25.359178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.359120    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.366297 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.366281    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.366353 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.366305    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-22.ec2.internal\": node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:25.380586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.380568    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-22.ec2.internal"]
Sep 08 14:16:25.380676 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.380635    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:16:25.382997 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.382982    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:16:25.383063 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.383026    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:16:25.383063 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.383036    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:16:25.385200 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.385186    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:16:25.385342 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.385328    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.385388 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.385355    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:16:25.385858 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.385839    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:16:25.385948 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.385883    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:16:25.385948 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.385896    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:16:25.385948 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.385904    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:16:25.385948 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.385932    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:16:25.385948 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.385945    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:16:25.386458 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.386440    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:25.387988 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.387970    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.388073 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.388000    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:16:25.388656 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.388642    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:16:25.388745 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.388670    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:16:25.388745 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.388684    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:16:25.421815 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.421790    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-22.ec2.internal\" not found" node="ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.426253 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.426238    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-22.ec2.internal\" not found" node="ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.482758 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.482732    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6d5a7600a30e525f429974d6c4f3b00c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal\" (UID: \"6d5a7600a30e525f429974d6c4f3b00c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.482839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.482761    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d5a7600a30e525f429974d6c4f3b00c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal\" (UID: \"6d5a7600a30e525f429974d6c4f3b00c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.482839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.482778    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aef2fcf38a14b047ca1c2b9fe741b9ea-config\") pod \"kube-apiserver-proxy-ip-10-0-131-22.ec2.internal\" (UID: \"aef2fcf38a14b047ca1c2b9fe741b9ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.486826 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.486809    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:25.583118 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.583075    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d5a7600a30e525f429974d6c4f3b00c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal\" (UID: \"6d5a7600a30e525f429974d6c4f3b00c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.583118 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.583117    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aef2fcf38a14b047ca1c2b9fe741b9ea-config\") pod \"kube-apiserver-proxy-ip-10-0-131-22.ec2.internal\" (UID: \"aef2fcf38a14b047ca1c2b9fe741b9ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.583296 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.583135    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6d5a7600a30e525f429974d6c4f3b00c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal\" (UID: \"6d5a7600a30e525f429974d6c4f3b00c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.583296 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.583191    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6d5a7600a30e525f429974d6c4f3b00c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal\" (UID: \"6d5a7600a30e525f429974d6c4f3b00c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.583296 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.583196    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d5a7600a30e525f429974d6c4f3b00c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal\" (UID: \"6d5a7600a30e525f429974d6c4f3b00c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.583296 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.583196    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aef2fcf38a14b047ca1c2b9fe741b9ea-config\") pod \"kube-apiserver-proxy-ip-10-0-131-22.ec2.internal\" (UID: \"aef2fcf38a14b047ca1c2b9fe741b9ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.587186 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.587171    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:25.688081 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.688014    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:25.724186 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.724163    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.729280 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:25.728965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:25.788273 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.788241    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:25.888824 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.888794    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:25.989441 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:25.989357    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:26.089651 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:26.089617    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:26.110304 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.110279    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 14:16:26.110442 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.110423    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 14:16:26.189863 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:26.189831    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:26.214586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.214549    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 14:11:25 +0000 UTC" deadline="2028-03-23 01:25:57.704395751 +0000 UTC"
Sep 08 14:16:26.214586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.214576    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13475h9m31.489823849s"
Sep 08 14:16:26.291153 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:26.290936    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:26.351404 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:26.351368    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d5a7600a30e525f429974d6c4f3b00c.slice/crio-37841518bbdd101a4877b9f9a268f5b3ea41366643f8c7d10c0b09753d9471f0 WatchSource:0}: Error finding container 37841518bbdd101a4877b9f9a268f5b3ea41366643f8c7d10c0b09753d9471f0: Status 404 returned error can't find the container with id 37841518bbdd101a4877b9f9a268f5b3ea41366643f8c7d10c0b09753d9471f0
Sep 08 14:16:26.355367 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.355354    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:16:26.361496 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.361474    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:16:26.391984 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:26.391960    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:26.492832 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:26.492801    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-22.ec2.internal\" not found"
Sep 08 14:16:26.523876 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.523850    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:16:26.582176 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.582100    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:26.599498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.599472    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 14:16:26.601838 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.601823    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-22.ec2.internal"
Sep 08 14:16:26.612000 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.611980    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 14:16:26.767502 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:26.767472    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:16:27.161114 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.161080    2558 apiserver.go:52] "Watching apiserver"
Sep 08 14:16:27.169675 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.169647    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 14:16:27.170129 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.170104    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-22.ec2.internal","openshift-cluster-node-tuning-operator/tuned-vq6c8","openshift-dns/node-resolver-shfmt","openshift-image-registry/node-ca-7c4rl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal","openshift-multus/multus-h2sbv","openshift-network-diagnostics/network-check-target-92v85","kube-system/global-pull-secret-syncer-drbnw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4","openshift-multus/multus-additional-cni-plugins-cqkch","openshift-multus/network-metrics-daemon-zcpxc","openshift-network-operator/iptables-alerter-w9dgr","openshift-ovn-kubernetes/ovnkube-node-4qb4z","kube-system/konnectivity-agent-88pdh"]
Sep 08 14:16:27.172864 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.172834    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:27.172989 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.172919    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:27.175379 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.175357    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7c4rl"
Sep 08 14:16:27.177671 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.177643    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:27.177817 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.177791    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:27.178341 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.178236    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vzjrl\""
Sep 08 14:16:27.178341 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.178252    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 14:16:27.178341 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.178271    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 14:16:27.178539 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.178425    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 14:16:27.179959 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.179940    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-shfmt"
Sep 08 14:16:27.181006 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.180987    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 14:16:27.182246 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.182225    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.182424 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.182405    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 14:16:27.182495 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.182460    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 14:16:27.182559 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.182496    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jffcq\""
Sep 08 14:16:27.184625 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.184601    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.186093 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.186069    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f9jmp\""
Sep 08 14:16:27.186337 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.186077    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 14:16:27.187125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.186670    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 14:16:27.187125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.186891    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 14:16:27.187125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.187111    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 14:16:27.187867 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.187758    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 14:16:27.187867 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.187785    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 14:16:27.188472 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.188451    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.188832 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.188622    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7cx8f\""
Sep 08 14:16:27.188906 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.188867    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 14:16:27.191048 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.191028    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.191153 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.191060    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 14:16:27.191975 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.191953    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 14:16:27.192088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.191960    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 14:16:27.192088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.192007    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8bjnm\""
Sep 08 14:16:27.192181 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.192135    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 14:16:27.192181 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.192153    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 14:16:27.192312 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.192289    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 14:16:27.193434 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.193411    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 14:16:27.193658 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.193639    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.193740 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.193719    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 14:16:27.194065 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194046    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 14:16:27.194145 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194046    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zvvrz\""
Sep 08 14:16:27.194641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194621    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23c51e8-9fb2-4e5c-883a-65b643ed3ef2-tmp-dir\") pod \"node-resolver-shfmt\" (UID: \"b23c51e8-9fb2-4e5c-883a-65b643ed3ef2\") " pod="openshift-dns/node-resolver-shfmt"
Sep 08 14:16:27.194690 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194655    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-cni-dir\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.194690 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194679    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-socket-dir-parent\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.194798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194714    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-run-k8s-cni-cncf-io\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.194798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194737    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-var-lib-cni-multus\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.194798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194760    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-daemon-config\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.194798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194793    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/859d3474-55bb-4163-b637-cf8a13e219d4-host\") pod \"node-ca-7c4rl\" (UID: \"859d3474-55bb-4163-b637-cf8a13e219d4\") " pod="openshift-image-registry/node-ca-7c4rl"
Sep 08 14:16:27.194947 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194829    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/859d3474-55bb-4163-b637-cf8a13e219d4-serviceca\") pod \"node-ca-7c4rl\" (UID: \"859d3474-55bb-4163-b637-cf8a13e219d4\") " pod="openshift-image-registry/node-ca-7c4rl"
Sep 08 14:16:27.194947 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194858    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tbkj\" (UniqueName: \"kubernetes.io/projected/859d3474-55bb-4163-b637-cf8a13e219d4-kube-api-access-9tbkj\") pod \"node-ca-7c4rl\" (UID: \"859d3474-55bb-4163-b637-cf8a13e219d4\") " pod="openshift-image-registry/node-ca-7c4rl"
Sep 08 14:16:27.194947 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-run-multus-certs\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.194947 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194891    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-etc-kubernetes\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.194947 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194908    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b23c51e8-9fb2-4e5c-883a-65b643ed3ef2-hosts-file\") pod \"node-resolver-shfmt\" (UID: \"b23c51e8-9fb2-4e5c-883a-65b643ed3ef2\") " pod="openshift-dns/node-resolver-shfmt"
Sep 08 14:16:27.194947 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194922    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-cnibin\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.194947 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194938    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-var-lib-kubelet\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.195292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.194972    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-hostroot\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.195292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.195051    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-conf-dir\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.195292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.195072    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msdqc\" (UniqueName: \"kubernetes.io/projected/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-kube-api-access-msdqc\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.195292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.195096    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:27.195292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.195126    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rg8r\" (UniqueName: \"kubernetes.io/projected/d96b871d-efd7-43e9-b268-978c310e94b4-kube-api-access-4rg8r\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:27.195292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.195158    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftvtn\" (UniqueName: \"kubernetes.io/projected/b23c51e8-9fb2-4e5c-883a-65b643ed3ef2-kube-api-access-ftvtn\") pod \"node-resolver-shfmt\" (UID: \"b23c51e8-9fb2-4e5c-883a-65b643ed3ef2\") " pod="openshift-dns/node-resolver-shfmt"
Sep 08 14:16:27.195292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.195190    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-system-cni-dir\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.195292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.195228    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-cni-binary-copy\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.195292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.195252    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-run-netns\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.195292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.195271    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-var-lib-cni-bin\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.195292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.195288    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ttwg\" (UniqueName: \"kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg\") pod \"network-check-target-92v85\" (UID: \"237de209-b858-44fd-ae88-a4ec84cf0d37\") " pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:27.196204 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.196185    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jdmbq\""
Sep 08 14:16:27.196352 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.196276    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w9dgr"
Sep 08 14:16:27.196426 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.196387    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 14:16:27.196514 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.196495    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 14:16:27.198772 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.198753    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:27.198908 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.198807    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:27.198985 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.198913    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 14:16:27.198985 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.198944    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 14:16:27.198985 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.198961    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gcdkf\""
Sep 08 14:16:27.199196 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.199182    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 14:16:27.201095 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.201077    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:27.203598 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.203580    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jt4sg\""
Sep 08 14:16:27.203693 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.203644    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 14:16:27.203693 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.203646    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 14:16:27.221922 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.221893    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t766d"
Sep 08 14:16:27.232374 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.232355    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t766d"
Sep 08 14:16:27.282363 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.282327    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 14:16:27.288548 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.288489    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal" event={"ID":"6d5a7600a30e525f429974d6c4f3b00c","Type":"ContainerStarted","Data":"37841518bbdd101a4877b9f9a268f5b3ea41366643f8c7d10c0b09753d9471f0"}
Sep 08 14:16:27.296071 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296041    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-cni-dir\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.296208 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-var-lib-cni-multus\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.296208 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296119    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-sys-fs\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.296208 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296146    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb26z\" (UniqueName: \"kubernetes.io/projected/0f419cc2-7833-4240-8628-3a0356e4f630-kube-api-access-zb26z\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.296208 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296174    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-cni-netd\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.296208 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296194    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-var-lib-cni-multus\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296199    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69cm8\" (UniqueName: \"kubernetes.io/projected/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-kube-api-access-69cm8\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296274    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-node-log\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296285    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-cni-dir\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296305    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-var-lib-kubelet\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296333    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-hostroot\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296338    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-var-lib-kubelet\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296359    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296371    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-hostroot\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296387    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-kubelet\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296414    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-var-lib-openvswitch\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-sys\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.296498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296467    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63c59478-7855-46c1-89ae-60de6aac1879-tmp\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296536    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftvtn\" (UniqueName: \"kubernetes.io/projected/b23c51e8-9fb2-4e5c-883a-65b643ed3ef2-kube-api-access-ftvtn\") pod \"node-resolver-shfmt\" (UID: \"b23c51e8-9fb2-4e5c-883a-65b643ed3ef2\") " pod="openshift-dns/node-resolver-shfmt"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296563    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-run-ovn\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296587    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-run-ovn-kubernetes\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296612    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gpgp\" (UniqueName: \"kubernetes.io/projected/7a70a048-18d4-4d02-a5d9-7660149a090d-kube-api-access-2gpgp\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296636    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-sysconfig\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.296646    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296659    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-var-lib-kubelet\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23c51e8-9fb2-4e5c-883a-65b643ed3ef2-tmp-dir\") pod \"node-resolver-shfmt\" (UID: \"b23c51e8-9fb2-4e5c-883a-65b643ed3ef2\") " pod="openshift-dns/node-resolver-shfmt"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296712    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-socket-dir-parent\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.296734    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs podName:d96b871d-efd7-43e9-b268-978c310e94b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:27.796717973 +0000 UTC m=+3.027649155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs") pod "network-metrics-daemon-zcpxc" (UID: "d96b871d-efd7-43e9-b268-978c310e94b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296765    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-run-k8s-cni-cncf-io\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296777    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-socket-dir-parent\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296784    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-modprobe-d\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296815    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f419cc2-7833-4240-8628-3a0356e4f630-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296878    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-etc-kubernetes\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.297098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296891    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-run-k8s-cni-cncf-io\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296917    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-log-socket\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-etc-kubernetes\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.296946    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a70a048-18d4-4d02-a5d9-7660149a090d-ovn-node-metrics-cert\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297007    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a70a048-18d4-4d02-a5d9-7660149a090d-ovnkube-script-lib\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297042    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297074    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rg8r\" (UniqueName: \"kubernetes.io/projected/d96b871d-efd7-43e9-b268-978c310e94b4-kube-api-access-4rg8r\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297096    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-system-cni-dir\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297122    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-cni-binary-copy\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297136    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23c51e8-9fb2-4e5c-883a-65b643ed3ef2-tmp-dir\") pod \"node-resolver-shfmt\" (UID: \"b23c51e8-9fb2-4e5c-883a-65b643ed3ef2\") " pod="openshift-dns/node-resolver-shfmt"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-var-lib-cni-bin\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297168    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttwg\" (UniqueName: \"kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg\") pod \"network-check-target-92v85\" (UID: \"237de209-b858-44fd-ae88-a4ec84cf0d37\") " pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297212    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-daemon-config\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297212    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-system-cni-dir\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297236    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-var-lib-cni-bin\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297241    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-run-multus-certs\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297325    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-run-multus-certs\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297366    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297400    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-cni-bin\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297426    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a70a048-18d4-4d02-a5d9-7660149a090d-ovnkube-config\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297456    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/859d3474-55bb-4163-b637-cf8a13e219d4-serviceca\") pod \"node-ca-7c4rl\" (UID: \"859d3474-55bb-4163-b637-cf8a13e219d4\") " pod="openshift-image-registry/node-ca-7c4rl"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297484    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tbkj\" (UniqueName: \"kubernetes.io/projected/859d3474-55bb-4163-b637-cf8a13e219d4-kube-api-access-9tbkj\") pod \"node-ca-7c4rl\" (UID: \"859d3474-55bb-4163-b637-cf8a13e219d4\") " pod="openshift-image-registry/node-ca-7c4rl"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297511    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e20eb2e-4a15-409a-9192-aafeec619b23-kubelet-config\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297537    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e20eb2e-4a15-409a-9192-aafeec619b23-dbus\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-systemd-units\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-run-openvswitch\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297664    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-sysctl-d\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297698    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-conf-dir\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297750    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-conf-dir\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297751    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msdqc\" (UniqueName: \"kubernetes.io/projected/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-kube-api-access-msdqc\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297789    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-socket-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297868    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-cni-binary-copy\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297874    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-etc-openvswitch\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297907    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-kubernetes\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.298839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297940    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-sysctl-conf\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297969    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/63c59478-7855-46c1-89ae-60de6aac1879-etc-tuned\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.297996    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-registration-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298045    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-run\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298058    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/859d3474-55bb-4163-b637-cf8a13e219d4-serviceca\") pod \"node-ca-7c4rl\" (UID: \"859d3474-55bb-4163-b637-cf8a13e219d4\") " pod="openshift-image-registry/node-ca-7c4rl"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298066    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-lib-modules\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298104    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f419cc2-7833-4240-8628-3a0356e4f630-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298141    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9efb8551-2b48-4721-8ac3-3b02ca1aa047-konnectivity-ca\") pod \"konnectivity-agent-88pdh\" (UID: \"9efb8551-2b48-4721-8ac3-3b02ca1aa047\") " pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298162    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-run-systemd\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298180    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dxn9\" (UniqueName: \"kubernetes.io/projected/63c59478-7855-46c1-89ae-60de6aac1879-kube-api-access-7dxn9\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298232    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/859d3474-55bb-4163-b637-cf8a13e219d4-host\") pod \"node-ca-7c4rl\" (UID: \"859d3474-55bb-4163-b637-cf8a13e219d4\") " pod="openshift-image-registry/node-ca-7c4rl"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298253    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/859d3474-55bb-4163-b637-cf8a13e219d4-host\") pod \"node-ca-7c4rl\" (UID: \"859d3474-55bb-4163-b637-cf8a13e219d4\") " pod="openshift-image-registry/node-ca-7c4rl"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298308    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-systemd\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298333    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-host\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298370    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f419cc2-7833-4240-8628-3a0356e4f630-cnibin\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298412    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f419cc2-7833-4240-8628-3a0356e4f630-cni-binary-copy\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.299365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298417    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-multus-daemon-config\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298436    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e3a57367-7de9-4541-9aae-766c14d4af8f-host-slash\") pod \"iptables-alerter-w9dgr\" (UID: \"e3a57367-7de9-4541-9aae-766c14d4af8f\") " pod="openshift-network-operator/iptables-alerter-w9dgr"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b23c51e8-9fb2-4e5c-883a-65b643ed3ef2-hosts-file\") pod \"node-resolver-shfmt\" (UID: \"b23c51e8-9fb2-4e5c-883a-65b643ed3ef2\") " pod="openshift-dns/node-resolver-shfmt"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-cnibin\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298522    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-cnibin\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298528    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-run-netns\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298533    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b23c51e8-9fb2-4e5c-883a-65b643ed3ef2-hosts-file\") pod \"node-resolver-shfmt\" (UID: \"b23c51e8-9fb2-4e5c-883a-65b643ed3ef2\") " pod="openshift-dns/node-resolver-shfmt"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f419cc2-7833-4240-8628-3a0356e4f630-system-cni-dir\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298585    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e3a57367-7de9-4541-9aae-766c14d4af8f-iptables-alerter-script\") pod \"iptables-alerter-w9dgr\" (UID: \"e3a57367-7de9-4541-9aae-766c14d4af8f\") " pod="openshift-network-operator/iptables-alerter-w9dgr"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298602    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdtpq\" (UniqueName: \"kubernetes.io/projected/e3a57367-7de9-4541-9aae-766c14d4af8f-kube-api-access-gdtpq\") pod \"iptables-alerter-w9dgr\" (UID: \"e3a57367-7de9-4541-9aae-766c14d4af8f\") " pod="openshift-network-operator/iptables-alerter-w9dgr"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298626    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-run-netns\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298651    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-host-run-netns\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298656    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9efb8551-2b48-4721-8ac3-3b02ca1aa047-agent-certs\") pod \"konnectivity-agent-88pdh\" (UID: \"9efb8551-2b48-4721-8ac3-3b02ca1aa047\") " pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298683    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-device-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298709    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-slash\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298734    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a70a048-18d4-4d02-a5d9-7660149a090d-env-overrides\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.299869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.298763    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f419cc2-7833-4240-8628-3a0356e4f630-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.308419 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.308399    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 14:16:27.312236 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.312208    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftvtn\" (UniqueName: \"kubernetes.io/projected/b23c51e8-9fb2-4e5c-883a-65b643ed3ef2-kube-api-access-ftvtn\") pod \"node-resolver-shfmt\" (UID: \"b23c51e8-9fb2-4e5c-883a-65b643ed3ef2\") " pod="openshift-dns/node-resolver-shfmt"
Sep 08 14:16:27.313552 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.313533    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:16:27.313552 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.313551    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:16:27.313720 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.313561    2558 projected.go:196] Error preparing data for projected volume kube-api-access-8ttwg for pod openshift-network-diagnostics/network-check-target-92v85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:27.313720 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.313608    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg podName:237de209-b858-44fd-ae88-a4ec84cf0d37 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:27.813594082 +0000 UTC m=+3.044525304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttwg" (UniqueName: "kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg") pod "network-check-target-92v85" (UID: "237de209-b858-44fd-ae88-a4ec84cf0d37") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:27.314128 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.314105    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rg8r\" (UniqueName: \"kubernetes.io/projected/d96b871d-efd7-43e9-b268-978c310e94b4-kube-api-access-4rg8r\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:27.315794 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.315773    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tbkj\" (UniqueName: \"kubernetes.io/projected/859d3474-55bb-4163-b637-cf8a13e219d4-kube-api-access-9tbkj\") pod \"node-ca-7c4rl\" (UID: \"859d3474-55bb-4163-b637-cf8a13e219d4\") " pod="openshift-image-registry/node-ca-7c4rl"
Sep 08 14:16:27.316463 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.316445    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msdqc\" (UniqueName: \"kubernetes.io/projected/b7cb6c46-c03d-4d43-b7ba-36dfacd49074-kube-api-access-msdqc\") pod \"multus-h2sbv\" (UID: \"b7cb6c46-c03d-4d43-b7ba-36dfacd49074\") " pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.399442 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399412    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a70a048-18d4-4d02-a5d9-7660149a090d-ovnkube-config\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.399629 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399453    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e20eb2e-4a15-409a-9192-aafeec619b23-kubelet-config\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:27.399629 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399469    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e20eb2e-4a15-409a-9192-aafeec619b23-dbus\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:27.399629 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399486    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-systemd-units\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.399629 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399543    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-systemd-units\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.399629 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399553    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e20eb2e-4a15-409a-9192-aafeec619b23-kubelet-config\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:27.399629 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399577    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-run-openvswitch\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.399629 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399617    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-sysctl-d\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399648    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-socket-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399658    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e20eb2e-4a15-409a-9192-aafeec619b23-dbus\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399669    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-run-openvswitch\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399677    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-etc-openvswitch\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-kubernetes\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399724    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-sysctl-conf\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399733    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-etc-openvswitch\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399744    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/63c59478-7855-46c1-89ae-60de6aac1879-etc-tuned\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399793    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-socket-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399799    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-sysctl-d\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399794    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-registration-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399843    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-registration-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399858    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-run\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399877    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-lib-modules\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399881    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-sysctl-conf\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399857    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-kubernetes\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399889    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-run\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399896    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f419cc2-7833-4240-8628-3a0356e4f630-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399943    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9efb8551-2b48-4721-8ac3-3b02ca1aa047-konnectivity-ca\") pod \"konnectivity-agent-88pdh\" (UID: \"9efb8551-2b48-4721-8ac3-3b02ca1aa047\") " pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399968    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-lib-modules\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399971    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-run-systemd\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.399997    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dxn9\" (UniqueName: \"kubernetes.io/projected/63c59478-7855-46c1-89ae-60de6aac1879-kube-api-access-7dxn9\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400049    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-systemd\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400076    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-host\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f419cc2-7833-4240-8628-3a0356e4f630-cnibin\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400115    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-systemd\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400133    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a70a048-18d4-4d02-a5d9-7660149a090d-ovnkube-config\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400138    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f419cc2-7833-4240-8628-3a0356e4f630-cni-binary-copy\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400169    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-host\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400050    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-run-systemd\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400197    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e3a57367-7de9-4541-9aae-766c14d4af8f-host-slash\") pod \"iptables-alerter-w9dgr\" (UID: \"e3a57367-7de9-4541-9aae-766c14d4af8f\") " pod="openshift-network-operator/iptables-alerter-w9dgr"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400234    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f419cc2-7833-4240-8628-3a0356e4f630-cnibin\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.400903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400236    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-run-netns\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400282    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f419cc2-7833-4240-8628-3a0356e4f630-system-cni-dir\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400274    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-run-netns\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400312    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e3a57367-7de9-4541-9aae-766c14d4af8f-host-slash\") pod \"iptables-alerter-w9dgr\" (UID: \"e3a57367-7de9-4541-9aae-766c14d4af8f\") " pod="openshift-network-operator/iptables-alerter-w9dgr"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400327    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f419cc2-7833-4240-8628-3a0356e4f630-system-cni-dir\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400323    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e3a57367-7de9-4541-9aae-766c14d4af8f-iptables-alerter-script\") pod \"iptables-alerter-w9dgr\" (UID: \"e3a57367-7de9-4541-9aae-766c14d4af8f\") " pod="openshift-network-operator/iptables-alerter-w9dgr"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400377    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdtpq\" (UniqueName: \"kubernetes.io/projected/e3a57367-7de9-4541-9aae-766c14d4af8f-kube-api-access-gdtpq\") pod \"iptables-alerter-w9dgr\" (UID: \"e3a57367-7de9-4541-9aae-766c14d4af8f\") " pod="openshift-network-operator/iptables-alerter-w9dgr"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9efb8551-2b48-4721-8ac3-3b02ca1aa047-agent-certs\") pod \"konnectivity-agent-88pdh\" (UID: \"9efb8551-2b48-4721-8ac3-3b02ca1aa047\") " pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400446    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-device-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400453    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-slash\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400498    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f419cc2-7833-4240-8628-3a0356e4f630-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400517    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-device-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a70a048-18d4-4d02-a5d9-7660149a090d-env-overrides\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400557    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f419cc2-7833-4240-8628-3a0356e4f630-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400566    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-slash\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400589    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-sys-fs\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.401855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400616    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb26z\" (UniqueName: \"kubernetes.io/projected/0f419cc2-7833-4240-8628-3a0356e4f630-kube-api-access-zb26z\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400658    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-cni-netd\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400702    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69cm8\" (UniqueName: \"kubernetes.io/projected/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-kube-api-access-69cm8\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400718    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-sys-fs\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400557    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9efb8551-2b48-4721-8ac3-3b02ca1aa047-konnectivity-ca\") pod \"konnectivity-agent-88pdh\" (UID: \"9efb8551-2b48-4721-8ac3-3b02ca1aa047\") " pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400756    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-node-log\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400800    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-cni-netd\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400809    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-node-log\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400828    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400878    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400934    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a70a048-18d4-4d02-a5d9-7660149a090d-env-overrides\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.400986    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-kubelet\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401018    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-var-lib-openvswitch\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-sys\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401042    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e3a57367-7de9-4541-9aae-766c14d4af8f-iptables-alerter-script\") pod \"iptables-alerter-w9dgr\" (UID: \"e3a57367-7de9-4541-9aae-766c14d4af8f\") " pod="openshift-network-operator/iptables-alerter-w9dgr"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401070    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63c59478-7855-46c1-89ae-60de6aac1879-tmp\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401090    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f419cc2-7833-4240-8628-3a0356e4f630-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.402599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401113    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-run-ovn\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401141    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-kubelet\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401174    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-var-lib-openvswitch\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401158    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-run-ovn-kubernetes\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401198    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-run-ovn-kubernetes\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401218    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gpgp\" (UniqueName: \"kubernetes.io/projected/7a70a048-18d4-4d02-a5d9-7660149a090d-kube-api-access-2gpgp\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401225    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-sys\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401247    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-sysconfig\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401242    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-run-ovn\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-var-lib-kubelet\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401340    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-modprobe-d\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401349    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-sysconfig\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f419cc2-7833-4240-8628-3a0356e4f630-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401394    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f419cc2-7833-4240-8628-3a0356e4f630-cni-binary-copy\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401405    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-var-lib-kubelet\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401397    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-log-socket\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401449    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-log-socket\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a70a048-18d4-4d02-a5d9-7660149a090d-ovn-node-metrics-cert\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401490    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a70a048-18d4-4d02-a5d9-7660149a090d-ovnkube-script-lib\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401517    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401522    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/63c59478-7855-46c1-89ae-60de6aac1879-etc-modprobe-d\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401603    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401608    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f419cc2-7833-4240-8628-3a0356e4f630-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401631    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-cni-bin\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401664    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.401700    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.401705    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a70a048-18d4-4d02-a5d9-7660149a090d-host-cni-bin\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.401789    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret podName:4e20eb2e-4a15-409a-9192-aafeec619b23 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:27.901768389 +0000 UTC m=+3.132699569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret") pod "global-pull-secret-syncer-drbnw" (UID: "4e20eb2e-4a15-409a-9192-aafeec619b23") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.402071    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a70a048-18d4-4d02-a5d9-7660149a090d-ovnkube-script-lib\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.402585    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/63c59478-7855-46c1-89ae-60de6aac1879-etc-tuned\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.403497    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9efb8551-2b48-4721-8ac3-3b02ca1aa047-agent-certs\") pod \"konnectivity-agent-88pdh\" (UID: \"9efb8551-2b48-4721-8ac3-3b02ca1aa047\") " pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:27.403809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.403691    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63c59478-7855-46c1-89ae-60de6aac1879-tmp\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.404261 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.403848    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a70a048-18d4-4d02-a5d9-7660149a090d-ovn-node-metrics-cert\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.414440 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.414375    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdtpq\" (UniqueName: \"kubernetes.io/projected/e3a57367-7de9-4541-9aae-766c14d4af8f-kube-api-access-gdtpq\") pod \"iptables-alerter-w9dgr\" (UID: \"e3a57367-7de9-4541-9aae-766c14d4af8f\") " pod="openshift-network-operator/iptables-alerter-w9dgr"
Sep 08 14:16:27.414619 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.414591    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69cm8\" (UniqueName: \"kubernetes.io/projected/87eaaa15-245c-4f32-b7bd-d31d9b7269b5-kube-api-access-69cm8\") pod \"aws-ebs-csi-driver-node-l8cs4\" (UID: \"87eaaa15-245c-4f32-b7bd-d31d9b7269b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.415230 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.415192    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dxn9\" (UniqueName: \"kubernetes.io/projected/63c59478-7855-46c1-89ae-60de6aac1879-kube-api-access-7dxn9\") pod \"tuned-vq6c8\" (UID: \"63c59478-7855-46c1-89ae-60de6aac1879\") " pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.416071 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.416052    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gpgp\" (UniqueName: \"kubernetes.io/projected/7a70a048-18d4-4d02-a5d9-7660149a090d-kube-api-access-2gpgp\") pod \"ovnkube-node-4qb4z\" (UID: \"7a70a048-18d4-4d02-a5d9-7660149a090d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.416153 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.416128    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb26z\" (UniqueName: \"kubernetes.io/projected/0f419cc2-7833-4240-8628-3a0356e4f630-kube-api-access-zb26z\") pod \"multus-additional-cni-plugins-cqkch\" (UID: \"0f419cc2-7833-4240-8628-3a0356e4f630\") " pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.417496 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.417477    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:16:27.487344 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.487306    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7c4rl"
Sep 08 14:16:27.497809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.497778    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-shfmt"
Sep 08 14:16:27.504690 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.504665    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h2sbv"
Sep 08 14:16:27.508990 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.508971    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4"
Sep 08 14:16:27.515821 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.515801    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:27.521661 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.521641    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vq6c8"
Sep 08 14:16:27.529367 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.529349    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cqkch"
Sep 08 14:16:27.538203 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.538184    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w9dgr"
Sep 08 14:16:27.544028 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.544007    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:27.758131 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:27.758082    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23c51e8_9fb2_4e5c_883a_65b643ed3ef2.slice/crio-bb9a15514f32c2493dc24068487212debf0c4713051ac6b42c76b94a56678538 WatchSource:0}: Error finding container bb9a15514f32c2493dc24068487212debf0c4713051ac6b42c76b94a56678538: Status 404 returned error can't find the container with id bb9a15514f32c2493dc24068487212debf0c4713051ac6b42c76b94a56678538
Sep 08 14:16:27.758725 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:27.758695    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a70a048_18d4_4d02_a5d9_7660149a090d.slice/crio-60584598bf4191c6fb25c84664761c07250bb36219964c6d182d65e5e899f7cf WatchSource:0}: Error finding container 60584598bf4191c6fb25c84664761c07250bb36219964c6d182d65e5e899f7cf: Status 404 returned error can't find the container with id 60584598bf4191c6fb25c84664761c07250bb36219964c6d182d65e5e899f7cf
Sep 08 14:16:27.762172 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:27.762147    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod859d3474_55bb_4163_b637_cf8a13e219d4.slice/crio-43dcbaebf8553ce149d3ccd1c62a1dbec4334e3429d6839522447acb5a003e90 WatchSource:0}: Error finding container 43dcbaebf8553ce149d3ccd1c62a1dbec4334e3429d6839522447acb5a003e90: Status 404 returned error can't find the container with id 43dcbaebf8553ce149d3ccd1c62a1dbec4334e3429d6839522447acb5a003e90
Sep 08 14:16:27.764147 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:27.764101    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efb8551_2b48_4721_8ac3_3b02ca1aa047.slice/crio-7afe8cc1cb883b08c56e813df1d9d53209cede2f47ec1db5526e3724d41db04e WatchSource:0}: Error finding container 7afe8cc1cb883b08c56e813df1d9d53209cede2f47ec1db5526e3724d41db04e: Status 404 returned error can't find the container with id 7afe8cc1cb883b08c56e813df1d9d53209cede2f47ec1db5526e3724d41db04e
Sep 08 14:16:27.765087 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:27.765063    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87eaaa15_245c_4f32_b7bd_d31d9b7269b5.slice/crio-da0a669522d6e25e3aa7c525c7b94c028f70899ea4643e247bfde748e41b9cdb WatchSource:0}: Error finding container da0a669522d6e25e3aa7c525c7b94c028f70899ea4643e247bfde748e41b9cdb: Status 404 returned error can't find the container with id da0a669522d6e25e3aa7c525c7b94c028f70899ea4643e247bfde748e41b9cdb
Sep 08 14:16:27.767011 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:27.766340    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63c59478_7855_46c1_89ae_60de6aac1879.slice/crio-c0959fb5e9c1deab178616bf041906c1de2e6c609a3662a45c74651c63857896 WatchSource:0}: Error finding container c0959fb5e9c1deab178616bf041906c1de2e6c609a3662a45c74651c63857896: Status 404 returned error can't find the container with id c0959fb5e9c1deab178616bf041906c1de2e6c609a3662a45c74651c63857896
Sep 08 14:16:27.805318 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.805262    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:27.805455 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.805437    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:27.805503 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.805493    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs podName:d96b871d-efd7-43e9-b268-978c310e94b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:28.80547938 +0000 UTC m=+4.036410557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs") pod "network-metrics-daemon-zcpxc" (UID: "d96b871d-efd7-43e9-b268-978c310e94b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:27.817711 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:16:27.817685    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef2fcf38a14b047ca1c2b9fe741b9ea.slice/crio-fc9800b02cff7f762b4484f587e69297d6daee2ed71645f8ada62af1f84b1d22 WatchSource:0}: Error finding container fc9800b02cff7f762b4484f587e69297d6daee2ed71645f8ada62af1f84b1d22: Status 404 returned error can't find the container with id fc9800b02cff7f762b4484f587e69297d6daee2ed71645f8ada62af1f84b1d22
Sep 08 14:16:27.906479 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.906437    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttwg\" (UniqueName: \"kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg\") pod \"network-check-target-92v85\" (UID: \"237de209-b858-44fd-ae88-a4ec84cf0d37\") " pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:27.906479 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:27.906485    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:27.906748 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.906601    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:27.906748 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.906641    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:16:27.906748 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.906663    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret podName:4e20eb2e-4a15-409a-9192-aafeec619b23 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:28.906644764 +0000 UTC m=+4.137575941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret") pod "global-pull-secret-syncer-drbnw" (UID: "4e20eb2e-4a15-409a-9192-aafeec619b23") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:27.906748 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.906672    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:16:27.906748 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.906687    2558 projected.go:196] Error preparing data for projected volume kube-api-access-8ttwg for pod openshift-network-diagnostics/network-check-target-92v85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:27.906748 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:27.906748    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg podName:237de209-b858-44fd-ae88-a4ec84cf0d37 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:28.906729071 +0000 UTC m=+4.137660250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttwg" (UniqueName: "kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg") pod "network-check-target-92v85" (UID: "237de209-b858-44fd-ae88-a4ec84cf0d37") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:28.236263 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.236168    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 14:11:27 +0000 UTC" deadline="2028-05-25 23:28:07.904662976 +0000 UTC"
Sep 08 14:16:28.236263 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.236257    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15009h11m39.668410268s"
Sep 08 14:16:28.291451 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.291416    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-88pdh" event={"ID":"9efb8551-2b48-4721-8ac3-3b02ca1aa047","Type":"ContainerStarted","Data":"7afe8cc1cb883b08c56e813df1d9d53209cede2f47ec1db5526e3724d41db04e"}
Sep 08 14:16:28.292619 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.292596    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7c4rl" event={"ID":"859d3474-55bb-4163-b637-cf8a13e219d4","Type":"ContainerStarted","Data":"43dcbaebf8553ce149d3ccd1c62a1dbec4334e3429d6839522447acb5a003e90"}
Sep 08 14:16:28.293739 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.293718    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z" event={"ID":"7a70a048-18d4-4d02-a5d9-7660149a090d","Type":"ContainerStarted","Data":"60584598bf4191c6fb25c84664761c07250bb36219964c6d182d65e5e899f7cf"}
Sep 08 14:16:28.294878 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.294852    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-shfmt" event={"ID":"b23c51e8-9fb2-4e5c-883a-65b643ed3ef2","Type":"ContainerStarted","Data":"bb9a15514f32c2493dc24068487212debf0c4713051ac6b42c76b94a56678538"}
Sep 08 14:16:28.295928 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.295906    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-22.ec2.internal" event={"ID":"aef2fcf38a14b047ca1c2b9fe741b9ea","Type":"ContainerStarted","Data":"fc9800b02cff7f762b4484f587e69297d6daee2ed71645f8ada62af1f84b1d22"}
Sep 08 14:16:28.297004 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.296983    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h2sbv" event={"ID":"b7cb6c46-c03d-4d43-b7ba-36dfacd49074","Type":"ContainerStarted","Data":"b8eb057443bfc55f34112ed61a3002fb7ce5aed860278dbcd6e5227bb202b390"}
Sep 08 14:16:28.298125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.298104    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w9dgr" event={"ID":"e3a57367-7de9-4541-9aae-766c14d4af8f","Type":"ContainerStarted","Data":"db9e06fad493e7003aaa898f2ed45ef2325b9d654a94d07c55524ca3cb469117"}
Sep 08 14:16:28.299338 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.299315    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cqkch" event={"ID":"0f419cc2-7833-4240-8628-3a0356e4f630","Type":"ContainerStarted","Data":"6f4f2f2e6266e81b7e61d10f90d63b8587e07fb43ff4890608a6fbf090790e30"}
Sep 08 14:16:28.300331 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.300303    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vq6c8" event={"ID":"63c59478-7855-46c1-89ae-60de6aac1879","Type":"ContainerStarted","Data":"c0959fb5e9c1deab178616bf041906c1de2e6c609a3662a45c74651c63857896"}
Sep 08 14:16:28.301282 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.301264    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4" event={"ID":"87eaaa15-245c-4f32-b7bd-d31d9b7269b5","Type":"ContainerStarted","Data":"da0a669522d6e25e3aa7c525c7b94c028f70899ea4643e247bfde748e41b9cdb"}
Sep 08 14:16:28.303137 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.303107    2558 generic.go:356] "Generic (PLEG): container finished" podID="6d5a7600a30e525f429974d6c4f3b00c" containerID="c6e0218a52119dc465b78eddb099bcb02516a943637063f35d6241db78b0ee29" exitCode=0
Sep 08 14:16:28.303221 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.303141    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal" event={"ID":"6d5a7600a30e525f429974d6c4f3b00c","Type":"ContainerDied","Data":"c6e0218a52119dc465b78eddb099bcb02516a943637063f35d6241db78b0ee29"}
Sep 08 14:16:28.814577 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.814481    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:28.814755 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:28.814680    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:28.814755 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:28.814746    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs podName:d96b871d-efd7-43e9-b268-978c310e94b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:30.814727577 +0000 UTC m=+6.045658756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs") pod "network-metrics-daemon-zcpxc" (UID: "d96b871d-efd7-43e9-b268-978c310e94b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:28.915025 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.914983    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttwg\" (UniqueName: \"kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg\") pod \"network-check-target-92v85\" (UID: \"237de209-b858-44fd-ae88-a4ec84cf0d37\") " pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:28.915206 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:28.915040    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:28.915270 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:28.915217    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:28.915362 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:28.915300    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret podName:4e20eb2e-4a15-409a-9192-aafeec619b23 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:30.915281398 +0000 UTC m=+6.146212577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret") pod "global-pull-secret-syncer-drbnw" (UID: "4e20eb2e-4a15-409a-9192-aafeec619b23") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:28.915744 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:28.915720    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:16:28.915825 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:28.915751    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:16:28.915825 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:28.915765    2558 projected.go:196] Error preparing data for projected volume kube-api-access-8ttwg for pod openshift-network-diagnostics/network-check-target-92v85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:28.915825 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:28.915810    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg podName:237de209-b858-44fd-ae88-a4ec84cf0d37 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:30.915795255 +0000 UTC m=+6.146726435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttwg" (UniqueName: "kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg") pod "network-check-target-92v85" (UID: "237de209-b858-44fd-ae88-a4ec84cf0d37") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:29.237477 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:29.237377    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 14:11:27 +0000 UTC" deadline="2028-03-20 05:29:41.56585021 +0000 UTC"
Sep 08 14:16:29.237477 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:29.237423    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13407h13m12.328431871s"
Sep 08 14:16:29.285002 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:29.284485    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:29.285002 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:29.284485    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:29.285002 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:29.284638    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:29.285002 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:29.284704    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:29.285002 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:29.284492    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:29.285002 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:29.284809    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:29.321030 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:29.320259    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal" event={"ID":"6d5a7600a30e525f429974d6c4f3b00c","Type":"ContainerStarted","Data":"44c102dc2a4b2db98f06bd0f7beee6e47d33e301fbea53b3410518b60d22ddbd"}
Sep 08 14:16:30.832893 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:30.832603    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:30.833432 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:30.832858    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:30.833432 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:30.833046    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs podName:d96b871d-efd7-43e9-b268-978c310e94b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:34.833024415 +0000 UTC m=+10.063955596 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs") pod "network-metrics-daemon-zcpxc" (UID: "d96b871d-efd7-43e9-b268-978c310e94b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:30.934929 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:30.934033    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttwg\" (UniqueName: \"kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg\") pod \"network-check-target-92v85\" (UID: \"237de209-b858-44fd-ae88-a4ec84cf0d37\") " pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:30.934929 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:30.934085    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:30.934929 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:30.934286    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:30.934929 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:30.934353    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret podName:4e20eb2e-4a15-409a-9192-aafeec619b23 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:34.934334305 +0000 UTC m=+10.165265486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret") pod "global-pull-secret-syncer-drbnw" (UID: "4e20eb2e-4a15-409a-9192-aafeec619b23") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:30.934929 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:30.934799    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:16:30.934929 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:30.934822    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:16:30.934929 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:30.934836    2558 projected.go:196] Error preparing data for projected volume kube-api-access-8ttwg for pod openshift-network-diagnostics/network-check-target-92v85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:30.934929 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:30.934891    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg podName:237de209-b858-44fd-ae88-a4ec84cf0d37 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:34.934866423 +0000 UTC m=+10.165797601 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttwg" (UniqueName: "kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg") pod "network-check-target-92v85" (UID: "237de209-b858-44fd-ae88-a4ec84cf0d37") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:31.281772 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:31.281674    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:31.281990 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:31.281674    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:31.281990 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:31.281829    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:31.281990 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:31.281821    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:31.281990 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:31.281930    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:31.282223 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:31.282020    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:33.281037 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:33.280951    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:33.281037 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:33.280980    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:33.281037 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:33.280982    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:33.281608 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:33.281074    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:33.281608 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:33.281198    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:33.281608 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:33.281299    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:34.865030 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:34.864959    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:34.865482 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:34.865109    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:34.865482 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:34.865182    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs podName:d96b871d-efd7-43e9-b268-978c310e94b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:42.865160651 +0000 UTC m=+18.096091831 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs") pod "network-metrics-daemon-zcpxc" (UID: "d96b871d-efd7-43e9-b268-978c310e94b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:34.966107 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:34.966064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttwg\" (UniqueName: \"kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg\") pod \"network-check-target-92v85\" (UID: \"237de209-b858-44fd-ae88-a4ec84cf0d37\") " pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:34.966301 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:34.966115    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:34.966301 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:34.966270    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:34.966408 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:34.966340    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret podName:4e20eb2e-4a15-409a-9192-aafeec619b23 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:42.966322722 +0000 UTC m=+18.197253898 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret") pod "global-pull-secret-syncer-drbnw" (UID: "4e20eb2e-4a15-409a-9192-aafeec619b23") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:34.966408 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:34.966272    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:16:34.966514 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:34.966408    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:16:34.966514 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:34.966429    2558 projected.go:196] Error preparing data for projected volume kube-api-access-8ttwg for pod openshift-network-diagnostics/network-check-target-92v85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:34.966514 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:34.966496    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg podName:237de209-b858-44fd-ae88-a4ec84cf0d37 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:42.966476839 +0000 UTC m=+18.197408037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttwg" (UniqueName: "kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg") pod "network-check-target-92v85" (UID: "237de209-b858-44fd-ae88-a4ec84cf0d37") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:35.281917 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:35.281813    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:35.282066 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:35.281944    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:35.282132 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:35.282119    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:35.282245 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:35.282201    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:35.282309 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:35.282260    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:35.282368 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:35.282328    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:37.281556 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:37.280975    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:37.281556 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:37.281109    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:37.281556 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:37.280975    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:37.281556 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:37.281457    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:37.281556 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:37.281485    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:37.282100 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:37.281579    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:39.281194 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:39.281157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:39.281691 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:39.281157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:39.281691 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:39.281308    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:39.281691 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:39.281169    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:39.281691 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:39.281380    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:39.281691 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:39.281439    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:41.280614 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:41.280577    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:41.281062 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:41.280577    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:41.281062 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:41.280722    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:41.281062 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:41.280746    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:41.281062 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:41.280901    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:41.281062 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:41.280993    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:42.924347 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:42.924305    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:42.924777 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:42.924455    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:42.924777 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:42.924522    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs podName:d96b871d-efd7-43e9-b268-978c310e94b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:58.92450699 +0000 UTC m=+34.155438167 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs") pod "network-metrics-daemon-zcpxc" (UID: "d96b871d-efd7-43e9-b268-978c310e94b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:43.025437 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:43.025356    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttwg\" (UniqueName: \"kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg\") pod \"network-check-target-92v85\" (UID: \"237de209-b858-44fd-ae88-a4ec84cf0d37\") " pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:43.025437 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:43.025402    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:43.025628 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:43.025553    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:16:43.025628 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:43.025583    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:16:43.025628 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:43.025587    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:43.025628 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:43.025594    2558 projected.go:196] Error preparing data for projected volume kube-api-access-8ttwg for pod openshift-network-diagnostics/network-check-target-92v85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:43.025778 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:43.025646    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret podName:4e20eb2e-4a15-409a-9192-aafeec619b23 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:59.025628503 +0000 UTC m=+34.256559688 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret") pod "global-pull-secret-syncer-drbnw" (UID: "4e20eb2e-4a15-409a-9192-aafeec619b23") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:43.025778 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:43.025664    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg podName:237de209-b858-44fd-ae88-a4ec84cf0d37 nodeName:}" failed. No retries permitted until 2026-09-08 14:16:59.025657807 +0000 UTC m=+34.256588994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttwg" (UniqueName: "kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg") pod "network-check-target-92v85" (UID: "237de209-b858-44fd-ae88-a4ec84cf0d37") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:43.280959 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:43.280874    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:43.280959 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:43.280905    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:43.281174 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:43.280990    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:43.281174 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:43.281009    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:43.281174 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:43.281105    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:43.281333 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:43.281180    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:45.281485 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:45.281452    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:45.281885 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:45.281569    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:45.281885 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:45.281627    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:45.281885 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:45.281658    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:45.281885 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:45.281704    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:45.281885 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:45.281783    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:47.281811 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.281471    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:47.282682 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.281481    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:47.282682 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:47.281844    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:47.282682 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.281481    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:47.282682 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:47.281899    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:47.282682 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:47.281984    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:47.367359 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.367324    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z" event={"ID":"7a70a048-18d4-4d02-a5d9-7660149a090d","Type":"ContainerStarted","Data":"b35e10b051298a5907e22b5ae4cfb64f4fa23b46ae7ad21eb08fed76a023a394"}
Sep 08 14:16:47.367531 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.367401    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z" event={"ID":"7a70a048-18d4-4d02-a5d9-7660149a090d","Type":"ContainerStarted","Data":"254402e572ca159365c2ff9ef42d1fecbc3a0e69838b7910dc6a522d1e5da824"}
Sep 08 14:16:47.367531 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.367446    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z" event={"ID":"7a70a048-18d4-4d02-a5d9-7660149a090d","Type":"ContainerStarted","Data":"ef2549ec87a6be51534a5dc3c757de3c390bc34103f55f019f01c194508ea043"}
Sep 08 14:16:47.367531 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.367490    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z" event={"ID":"7a70a048-18d4-4d02-a5d9-7660149a090d","Type":"ContainerStarted","Data":"cf9111c79bfab5d8d17d079c532b93aaa115ce9767a12dc026b3e86a0f149aea"}
Sep 08 14:16:47.367670 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.367532    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z" event={"ID":"7a70a048-18d4-4d02-a5d9-7660149a090d","Type":"ContainerStarted","Data":"d9e6c09fb4d662f5b4358c5879334c3f54e9d476d3890c33959ff21bb1fc0fd2"}
Sep 08 14:16:47.367670 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.367580    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z" event={"ID":"7a70a048-18d4-4d02-a5d9-7660149a090d","Type":"ContainerStarted","Data":"497cfcea1f04d38fd0f7dfb7ae31e058456894126cd09e9b1f38ea6b6f6f2636"}
Sep 08 14:16:47.368749 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.368728    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-shfmt" event={"ID":"b23c51e8-9fb2-4e5c-883a-65b643ed3ef2","Type":"ContainerStarted","Data":"f841beea1236039ed6fe8c743a99e2272699623ff69c6ef9fbc23a7d7f24301e"}
Sep 08 14:16:47.370138 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.370113    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-22.ec2.internal" event={"ID":"aef2fcf38a14b047ca1c2b9fe741b9ea","Type":"ContainerStarted","Data":"aa61e4c3a8e0dd0e4104ad9d633d7b3410c572f70d0028f6cdc64ea594012662"}
Sep 08 14:16:47.371461 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.371438    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h2sbv" event={"ID":"b7cb6c46-c03d-4d43-b7ba-36dfacd49074","Type":"ContainerStarted","Data":"7380dfcf7f6544a26b6c0bcd2538854e6d8de1344ebd670b1371760f83382f04"}
Sep 08 14:16:47.372806 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.372785    2558 generic.go:356] "Generic (PLEG): container finished" podID="0f419cc2-7833-4240-8628-3a0356e4f630" containerID="384ff48fe30a0cde6b46ef82a8e030254261dbb5081e17fcd3e0ef1166d3d866" exitCode=0
Sep 08 14:16:47.372896 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.372855    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cqkch" event={"ID":"0f419cc2-7833-4240-8628-3a0356e4f630","Type":"ContainerDied","Data":"384ff48fe30a0cde6b46ef82a8e030254261dbb5081e17fcd3e0ef1166d3d866"}
Sep 08 14:16:47.374365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.374345    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vq6c8" event={"ID":"63c59478-7855-46c1-89ae-60de6aac1879","Type":"ContainerStarted","Data":"aaf86ba6440caac86f26adc8645cb03945eccead569ddaa2f5fad491604645a9"}
Sep 08 14:16:47.377488 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.377466    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4" event={"ID":"87eaaa15-245c-4f32-b7bd-d31d9b7269b5","Type":"ContainerStarted","Data":"7fa6e05c554f3479a6ae9df45f66eb23845ed7f8b134eacfbaf6b1d16e79ed23"}
Sep 08 14:16:47.379146 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.379125    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-88pdh" event={"ID":"9efb8551-2b48-4721-8ac3-3b02ca1aa047","Type":"ContainerStarted","Data":"bffb9df02155d1a53de84d1e280706577c0ead705dc35a3765f065266996a55a"}
Sep 08 14:16:47.380772 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.380751    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7c4rl" event={"ID":"859d3474-55bb-4163-b637-cf8a13e219d4","Type":"ContainerStarted","Data":"cd7b148f8159853f111938eaa6c40cbe81c12cf3a077ac3a816f4ea314356eef"}
Sep 08 14:16:47.386115 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.386077    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-shfmt" podStartSLOduration=2.6603980050000002 podStartE2EDuration="21.386067573s" totalImagesPullingTime="18.725669568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:16:47.385185099 +0000 UTC m=+22.616116306" watchObservedRunningTime="2026-09-08 14:16:47.386067573 +0000 UTC m=+22.616998761"
Sep 08 14:16:47.386256 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.386231    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-22.ec2.internal" podStartSLOduration=20.386224126 podStartE2EDuration="20.386224126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:16:29.341581404 +0000 UTC m=+4.572512606" watchObservedRunningTime="2026-09-08 14:16:47.386224126 +0000 UTC m=+22.617155315"
Sep 08 14:16:47.400726 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.400692    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-88pdh" podStartSLOduration=2.685489433 podStartE2EDuration="21.400680202s" totalImagesPullingTime="18.715190769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:16:47.400032503 +0000 UTC m=+22.630963701" watchObservedRunningTime="2026-09-08 14:16:47.400680202 +0000 UTC m=+22.631611399"
Sep 08 14:16:47.414269 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.414221    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-22.ec2.internal" podStartSLOduration=20.414187945 podStartE2EDuration="20.414187945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:16:47.413867319 +0000 UTC m=+22.644798526" watchObservedRunningTime="2026-09-08 14:16:47.414187945 +0000 UTC m=+22.645119144"
Sep 08 14:16:47.429270 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.429230    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7c4rl" podStartSLOduration=3.707192987 podStartE2EDuration="22.42920076s" totalImagesPullingTime="18.722007773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:16:47.428752577 +0000 UTC m=+22.659683774" watchObservedRunningTime="2026-09-08 14:16:47.42920076 +0000 UTC m=+22.660131956"
Sep 08 14:16:47.446286 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.446248    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h2sbv" podStartSLOduration=2.684717403 podStartE2EDuration="21.446236215s" totalImagesPullingTime="18.761518812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:16:47.446029932 +0000 UTC m=+22.676961130" watchObservedRunningTime="2026-09-08 14:16:47.446236215 +0000 UTC m=+22.677167411"
Sep 08 14:16:47.476053 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:47.473480    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vq6c8" podStartSLOduration=2.757569707 podStartE2EDuration="21.473460456s" totalImagesPullingTime="18.715890749s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:16:47.472945134 +0000 UTC m=+22.703876359" watchObservedRunningTime="2026-09-08 14:16:47.473460456 +0000 UTC m=+22.704391654"
Sep 08 14:16:48.068153 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:48.068119    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 14:16:48.270619 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:48.270462    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T14:16:48.068142269Z","UUID":"077d9e8a-7abd-46d6-b578-cc16e7879eee","Handler":null,"Name":"","Endpoint":""}
Sep 08 14:16:48.271834 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:48.271818    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 14:16:48.271919 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:48.271846    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 14:16:48.384903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:48.384864    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w9dgr" event={"ID":"e3a57367-7de9-4541-9aae-766c14d4af8f","Type":"ContainerStarted","Data":"1362121b540feef53d37bc784b99ed508e1dae289871404322b9323e4f34176b"}
Sep 08 14:16:48.385586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:48.385563    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:48.386128 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:48.386109    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:48.386865 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:48.386842    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4" event={"ID":"87eaaa15-245c-4f32-b7bd-d31d9b7269b5","Type":"ContainerStarted","Data":"9a64a51039032df9919540b22f28b1eb81fd1959b95dd8856b09baecd7c209f5"}
Sep 08 14:16:48.387341 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:48.387305    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:48.387747 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:48.387730    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-88pdh"
Sep 08 14:16:48.403228 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:48.403175    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w9dgr" podStartSLOduration=3.715040394 podStartE2EDuration="22.403153068s" totalImagesPullingTime="18.688112674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:16:48.402710452 +0000 UTC m=+23.633641651" watchObservedRunningTime="2026-09-08 14:16:48.403153068 +0000 UTC m=+23.634084292"
Sep 08 14:16:49.281514 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:49.281485    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:49.281661 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:49.281485    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:49.281661 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:49.281609    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:49.281761 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:49.281485    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:49.281761 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:49.281686    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:49.281830 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:49.281761    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:49.393960 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:49.393868    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z" event={"ID":"7a70a048-18d4-4d02-a5d9-7660149a090d","Type":"ContainerStarted","Data":"ea3220e548f95b7a2ce645c0d4b9692eb4096a753806652f539f62b8835185b3"}
Sep 08 14:16:50.398531 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:50.398491    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4" event={"ID":"87eaaa15-245c-4f32-b7bd-d31d9b7269b5","Type":"ContainerStarted","Data":"cd98d090e81b348524a7f04f405b951c18c30f9107a941898fdca9ad9e1e3568"}
Sep 08 14:16:50.416835 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:50.416784    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8cs4" podStartSLOduration=2.735171117 podStartE2EDuration="24.416766108s" totalImagesPullingTime="21.681594991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:16:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:16:50.416343556 +0000 UTC m=+25.647274753" watchObservedRunningTime="2026-09-08 14:16:50.416766108 +0000 UTC m=+25.647697305"
Sep 08 14:16:51.280966 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:51.280938    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:51.281121 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:51.281066    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:51.281121 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:51.280937    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:51.281254 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:51.281197    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:51.281254 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:51.281248    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:51.281372 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:51.281355    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:53.281620 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:53.281450    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:53.282575 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:53.281452    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:53.282575 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:53.281708    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:53.282575 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:53.281452    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:53.282575 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:53.281777    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:53.282575 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:53.281843    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:53.410367 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:53.410337    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z" event={"ID":"7a70a048-18d4-4d02-a5d9-7660149a090d","Type":"ContainerStarted","Data":"6369a2204253f9b55914b0ba522a084f60ccf3334daf82bb3d38f7c8496bb196"}
Sep 08 14:16:53.410580 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:53.410561    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:53.412290 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:53.412262    2558 generic.go:356] "Generic (PLEG): container finished" podID="0f419cc2-7833-4240-8628-3a0356e4f630" containerID="71b2546f686ef66c60f401ead7e3eb055a9122a0b9787c4792f7f058639e0c36" exitCode=0
Sep 08 14:16:53.412437 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:53.412314    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cqkch" event={"ID":"0f419cc2-7833-4240-8628-3a0356e4f630","Type":"ContainerDied","Data":"71b2546f686ef66c60f401ead7e3eb055a9122a0b9787c4792f7f058639e0c36"}
Sep 08 14:16:53.426488 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:53.426466    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:53.438606 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:53.438570    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z" podStartSLOduration=8.656809123 podStartE2EDuration="27.438557584s" totalImagesPullingTime="18.781748461s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:16:53.438232941 +0000 UTC m=+28.669164137" watchObservedRunningTime="2026-09-08 14:16:53.438557584 +0000 UTC m=+28.669488781"
Sep 08 14:16:54.387345 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:54.387129    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-drbnw"]
Sep 08 14:16:54.387708 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:54.387445    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:54.387708 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:54.387549    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:54.389684 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:54.389663    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zcpxc"]
Sep 08 14:16:54.389813 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:54.389763    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:54.389879 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:54.389862    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:54.399120 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:54.399097    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-92v85"]
Sep 08 14:16:54.399241 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:54.399220    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:54.399331 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:54.399312    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:54.415588 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:54.415559    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:54.415699 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:54.415649    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:54.435341 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:54.435313    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:16:55.419268 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:55.419231    2558 generic.go:356] "Generic (PLEG): container finished" podID="0f419cc2-7833-4240-8628-3a0356e4f630" containerID="ef602a2571982db42b1e6824396750468cf519c3847f223bc50c8d6df1411600" exitCode=0
Sep 08 14:16:55.419693 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:55.419315    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cqkch" event={"ID":"0f419cc2-7833-4240-8628-3a0356e4f630","Type":"ContainerDied","Data":"ef602a2571982db42b1e6824396750468cf519c3847f223bc50c8d6df1411600"}
Sep 08 14:16:56.280850 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:56.280753    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:56.280850 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:56.280804    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:56.281055 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:56.280755    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:56.281055 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:56.280881    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:56.281055 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:56.280951    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:56.281055 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:56.281037    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:57.425584 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:57.425545    2558 generic.go:356] "Generic (PLEG): container finished" podID="0f419cc2-7833-4240-8628-3a0356e4f630" containerID="b20a491641d25d6e036a6b5766c6dd6f49d57192d80bc09d51ab2c23094f8505" exitCode=0
Sep 08 14:16:57.426033 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:57.425597    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cqkch" event={"ID":"0f419cc2-7833-4240-8628-3a0356e4f630","Type":"ContainerDied","Data":"b20a491641d25d6e036a6b5766c6dd6f49d57192d80bc09d51ab2c23094f8505"}
Sep 08 14:16:58.281304 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:58.281232    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:58.281453 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:58.281232    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:58.281453 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:58.281335    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-drbnw" podUID="4e20eb2e-4a15-409a-9192-aafeec619b23"
Sep 08 14:16:58.281453 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:58.281232    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:58.281453 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:58.281428    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-92v85" podUID="237de209-b858-44fd-ae88-a4ec84cf0d37"
Sep 08 14:16:58.281597 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:58.281526    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcpxc" podUID="d96b871d-efd7-43e9-b268-978c310e94b4"
Sep 08 14:16:58.928474 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:58.928435    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:16:58.928862 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:58.928606    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:58.928862 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:58.928671    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs podName:d96b871d-efd7-43e9-b268-978c310e94b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:30.928654044 +0000 UTC m=+66.159585240 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs") pod "network-metrics-daemon-zcpxc" (UID: "d96b871d-efd7-43e9-b268-978c310e94b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:16:59.029285 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.029086    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttwg\" (UniqueName: \"kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg\") pod \"network-check-target-92v85\" (UID: \"237de209-b858-44fd-ae88-a4ec84cf0d37\") " pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:16:59.029440 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.029305    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:16:59.029440 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:59.029314    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:16:59.029440 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:59.029350    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:16:59.029440 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:59.029363    2558 projected.go:196] Error preparing data for projected volume kube-api-access-8ttwg for pod openshift-network-diagnostics/network-check-target-92v85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:59.029440 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:59.029429    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg podName:237de209-b858-44fd-ae88-a4ec84cf0d37 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:31.029409198 +0000 UTC m=+66.260340389 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttwg" (UniqueName: "kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg") pod "network-check-target-92v85" (UID: "237de209-b858-44fd-ae88-a4ec84cf0d37") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:16:59.029702 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:59.029463    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:59.029702 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:59.029528    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret podName:4e20eb2e-4a15-409a-9192-aafeec619b23 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:31.029512343 +0000 UTC m=+66.260443531 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret") pod "global-pull-secret-syncer-drbnw" (UID: "4e20eb2e-4a15-409a-9192-aafeec619b23") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:16:59.604011 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.603900    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-22.ec2.internal" event="NodeReady"
Sep 08 14:16:59.604241 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.604052    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 14:16:59.642295 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.642264    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-854976b669-4fdlh"]
Sep 08 14:16:59.698867 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.698835    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zz6lc"]
Sep 08 14:16:59.699034 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.698954    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.701770 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.701745    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 08 14:16:59.701770 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.701761    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kj4kg\""
Sep 08 14:16:59.701969 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.701784    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 08 14:16:59.702218 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.702187    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 08 14:16:59.708280 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.708259    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 08 14:16:59.709790 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.709769    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-854976b669-4fdlh"]
Sep 08 14:16:59.709884 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.709803    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gxn5d"]
Sep 08 14:16:59.709940 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.709892    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.713390 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.712574    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 14:16:59.713390 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.712612    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rlxm6\""
Sep 08 14:16:59.713390 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.713064    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 14:16:59.727773 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.727753    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zz6lc"]
Sep 08 14:16:59.727861 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.727792    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gxn5d"]
Sep 08 14:16:59.727861 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.727852    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gxn5d"
Sep 08 14:16:59.730382 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.730362    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 14:16:59.730476 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.730451    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-j8jbf\""
Sep 08 14:16:59.730535 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.730488    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 14:16:59.730586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.730571    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 14:16:59.823533 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.823504    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:16:59.834458 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834433    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc8wg\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-kube-api-access-fc8wg\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.834600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834467    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp9vt\" (UniqueName: \"kubernetes.io/projected/a1d1aacf-d0b7-4da2-bd12-beaefcc94966-kube-api-access-xp9vt\") pod \"ingress-canary-gxn5d\" (UID: \"a1d1aacf-d0b7-4da2-bd12-beaefcc94966\") " pod="openshift-ingress-canary/ingress-canary-gxn5d"
Sep 08 14:16:59.834600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834498    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-trusted-ca\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.834600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834520    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.834600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834541    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-tmp-dir\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.834600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834568    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f194f86-4302-46b1-b765-094007b9b1b4-ca-trust-extracted\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.834792 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834681    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-image-registry-private-configuration\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.834792 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-installation-pull-secrets\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.834792 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834737    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-config-volume\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.834792 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834767    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-888nd\" (UniqueName: \"kubernetes.io/projected/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-kube-api-access-888nd\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.834792 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834782    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1d1aacf-d0b7-4da2-bd12-beaefcc94966-cert\") pod \"ingress-canary-gxn5d\" (UID: \"a1d1aacf-d0b7-4da2-bd12-beaefcc94966\") " pod="openshift-ingress-canary/ingress-canary-gxn5d"
Sep 08 14:16:59.835020 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834806    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-registry-certificates\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.835020 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834873    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.835020 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.834903    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-bound-sa-token\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.928571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.928489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:16:59.935606 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935573    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-image-registry-private-configuration\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.935762 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935614    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-installation-pull-secrets\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.935762 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935632    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-config-volume\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.935762 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935688    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-888nd\" (UniqueName: \"kubernetes.io/projected/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-kube-api-access-888nd\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.935762 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935726    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1d1aacf-d0b7-4da2-bd12-beaefcc94966-cert\") pod \"ingress-canary-gxn5d\" (UID: \"a1d1aacf-d0b7-4da2-bd12-beaefcc94966\") " pod="openshift-ingress-canary/ingress-canary-gxn5d"
Sep 08 14:16:59.935762 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935761    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-registry-certificates\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.936021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935810    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.936021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935836    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-bound-sa-token\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.936021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc8wg\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-kube-api-access-fc8wg\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.936021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935908    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp9vt\" (UniqueName: \"kubernetes.io/projected/a1d1aacf-d0b7-4da2-bd12-beaefcc94966-kube-api-access-xp9vt\") pod \"ingress-canary-gxn5d\" (UID: \"a1d1aacf-d0b7-4da2-bd12-beaefcc94966\") " pod="openshift-ingress-canary/ingress-canary-gxn5d"
Sep 08 14:16:59.936021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935941    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-trusted-ca\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.936021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.935970    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.936300 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.936029    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-tmp-dir\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.936300 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.936068    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f194f86-4302-46b1-b765-094007b9b1b4-ca-trust-extracted\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.936300 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.936265    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-config-volume\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.936840 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.936488    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f194f86-4302-46b1-b765-094007b9b1b4-ca-trust-extracted\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.936840 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:59.936560    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:16:59.936840 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:59.936630    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls podName:ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:00.436611839 +0000 UTC m=+35.667543025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls") pod "dns-default-zz6lc" (UID: "ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1") : secret "dns-default-metrics-tls" not found
Sep 08 14:16:59.936840 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.936746    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-registry-certificates\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.936840 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.936757    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-tmp-dir\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.936840 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:59.936830    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 14:16:59.936840 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:59.936847    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854976b669-4fdlh: secret "image-registry-tls" not found
Sep 08 14:16:59.937186 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:16:59.936925    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls podName:8f194f86-4302-46b1-b765-094007b9b1b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:00.436899232 +0000 UTC m=+35.667830421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls") pod "image-registry-854976b669-4fdlh" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4") : secret "image-registry-tls" not found
Sep 08 14:16:59.937569 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.937543    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-trusted-ca\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.940402 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.940382    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-image-registry-private-configuration\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.940490 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.940398    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-installation-pull-secrets\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.940549 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.940530    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1d1aacf-d0b7-4da2-bd12-beaefcc94966-cert\") pod \"ingress-canary-gxn5d\" (UID: \"a1d1aacf-d0b7-4da2-bd12-beaefcc94966\") " pod="openshift-ingress-canary/ingress-canary-gxn5d"
Sep 08 14:16:59.948649 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.948619    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-bound-sa-token\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.949059 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.949039    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc8wg\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-kube-api-access-fc8wg\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:16:59.949290 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.949270    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-888nd\" (UniqueName: \"kubernetes.io/projected/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-kube-api-access-888nd\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:16:59.949426 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:16:59.949405    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp9vt\" (UniqueName: \"kubernetes.io/projected/a1d1aacf-d0b7-4da2-bd12-beaefcc94966-kube-api-access-xp9vt\") pod \"ingress-canary-gxn5d\" (UID: \"a1d1aacf-d0b7-4da2-bd12-beaefcc94966\") " pod="openshift-ingress-canary/ingress-canary-gxn5d"
Sep 08 14:17:00.036755 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.036717    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gxn5d"
Sep 08 14:17:00.281236 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.281155    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:17:00.281374 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.281265    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:17:00.281374 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.281297    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:17:00.283957 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.283931    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 14:17:00.284323 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.284303    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 14:17:00.284440 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.284308    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 14:17:00.284440 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.284343    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vvt7d\""
Sep 08 14:17:00.285077 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.285056    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zbnn7\""
Sep 08 14:17:00.285178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.285089    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 14:17:00.373367 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.373336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:00.378993 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.378968    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gxn5d"]
Sep 08 14:17:00.383393 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:00.383369    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d1aacf_d0b7_4da2_bd12_beaefcc94966.slice/crio-2a649790cc635da868248ea8313776abae3167692f86e175b8bc48f6000a294b WatchSource:0}: Error finding container 2a649790cc635da868248ea8313776abae3167692f86e175b8bc48f6000a294b: Status 404 returned error can't find the container with id 2a649790cc635da868248ea8313776abae3167692f86e175b8bc48f6000a294b
Sep 08 14:17:00.433948 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.433908    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gxn5d" event={"ID":"a1d1aacf-d0b7-4da2-bd12-beaefcc94966","Type":"ContainerStarted","Data":"2a649790cc635da868248ea8313776abae3167692f86e175b8bc48f6000a294b"}
Sep 08 14:17:00.439377 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.439344    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:17:00.439501 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.439403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:17:00.439566 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:00.439507    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 14:17:00.439566 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:00.439531    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854976b669-4fdlh: secret "image-registry-tls" not found
Sep 08 14:17:00.439668 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:00.439586    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls podName:8f194f86-4302-46b1-b765-094007b9b1b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:01.43956656 +0000 UTC m=+36.670497760 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls") pod "image-registry-854976b669-4fdlh" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4") : secret "image-registry-tls" not found
Sep 08 14:17:00.439668 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:00.439509    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:17:00.439768 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:00.439685    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls podName:ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:01.43965883 +0000 UTC m=+36.670590017 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls") pod "dns-default-zz6lc" (UID: "ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1") : secret "dns-default-metrics-tls" not found
Sep 08 14:17:00.470155 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.470119    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:00.573170 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:00.573091    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:01.121602 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:01.121569    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:01.177263 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:01.177228    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:01.426895 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:01.426813    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:01.447737 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:01.447702    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:17:01.447922 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:01.447779    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:17:01.447922 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:01.447884    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:17:01.448015 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:01.447961    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls podName:ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:03.447940851 +0000 UTC m=+38.678872031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls") pod "dns-default-zz6lc" (UID: "ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1") : secret "dns-default-metrics-tls" not found
Sep 08 14:17:01.448015 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:01.447894    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 14:17:01.448015 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:01.447991    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854976b669-4fdlh: secret "image-registry-tls" not found
Sep 08 14:17:01.448153 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:01.448066    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls podName:8f194f86-4302-46b1-b765-094007b9b1b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:03.448051172 +0000 UTC m=+38.678982367 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls") pod "image-registry-854976b669-4fdlh" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4") : secret "image-registry-tls" not found
Sep 08 14:17:01.526554 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:01.526519    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:02.218387 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.218356    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:02.308509 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.308475    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:02.760734 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.760487    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"]
Sep 08 14:17:02.774144 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.774119    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"
Sep 08 14:17:02.774303 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.774119    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-585d697d6-dm2q6"]
Sep 08 14:17:02.776754 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.776731    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 08 14:17:02.777093 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.777036    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 08 14:17:02.777093 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.777062    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 08 14:17:02.777277 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.777188    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 08 14:17:02.777277 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.777038    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pj644\""
Sep 08 14:17:02.788399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.788379    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"]
Sep 08 14:17:02.788490 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.788408    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-585d697d6-dm2q6"]
Sep 08 14:17:02.788536 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.788490    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.791092 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.791074    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 14:17:02.791215 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.791094    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 08 14:17:02.791292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.791244    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 08 14:17:02.791475 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.791360    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 08 14:17:02.791475 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.791387    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 08 14:17:02.791475 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.791396    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 14:17:02.791475 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.791394    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f6vv4\""
Sep 08 14:17:02.859275 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.859243    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"]
Sep 08 14:17:02.860623 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.860596    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3039d4d-4454-4649-8e1a-6dc0f5bfe51b-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cxl6r\" (UID: \"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"
Sep 08 14:17:02.860732 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.860642    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cltqh\" (UniqueName: \"kubernetes.io/projected/d3039d4d-4454-4649-8e1a-6dc0f5bfe51b-kube-api-access-cltqh\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cxl6r\" (UID: \"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"
Sep 08 14:17:02.860732 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.860693    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-stats-auth\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.860833 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.860776    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-default-certificate\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.860833 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.860814    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb526\" (UniqueName: \"kubernetes.io/projected/595b339a-11fd-45dc-a950-63dcd5236b02-kube-api-access-rb526\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.860914 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.860846    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.860914 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.860877    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3039d4d-4454-4649-8e1a-6dc0f5bfe51b-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cxl6r\" (UID: \"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"
Sep 08 14:17:02.860994 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.860949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.873018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.872986    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"]
Sep 08 14:17:02.873018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.873013    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"
Sep 08 14:17:02.876671 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.876034    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 08 14:17:02.876671 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.876390    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-knl6w\""
Sep 08 14:17:02.876671 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.876585    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 08 14:17:02.876904 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.876872    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 08 14:17:02.877354 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.877138    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 08 14:17:02.884048 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.884026    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-chktm"]
Sep 08 14:17:02.884572 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.884550    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:02.887180 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.887148    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 08 14:17:02.887285 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.887200    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 08 14:17:02.887357 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.887324    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 08 14:17:02.887411 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.887372    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zvvdc\""
Sep 08 14:17:02.896812 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.896792    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-chktm"]
Sep 08 14:17:02.896909 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.896830    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"]
Sep 08 14:17:02.896909 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.896856    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"]
Sep 08 14:17:02.897039 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.897009    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-chktm"
Sep 08 14:17:02.900097 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.900051    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-v6j5g\""
Sep 08 14:17:02.935426 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.935393    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:02.961794 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.961765    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3039d4d-4454-4649-8e1a-6dc0f5bfe51b-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cxl6r\" (UID: \"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"
Sep 08 14:17:02.961956 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.961805    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cltqh\" (UniqueName: \"kubernetes.io/projected/d3039d4d-4454-4649-8e1a-6dc0f5bfe51b-kube-api-access-cltqh\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cxl6r\" (UID: \"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"
Sep 08 14:17:02.961956 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.961834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-stats-auth\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.961956 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.961931    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-default-certificate\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.962108 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.961976    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb526\" (UniqueName: \"kubernetes.io/projected/595b339a-11fd-45dc-a950-63dcd5236b02-kube-api-access-rb526\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.962108 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.962010    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fc3293c-9e19-4a82-b33b-dea02080290f-serving-cert\") pod \"service-ca-operator-859f94c4b-rn2gb\" (UID: \"3fc3293c-9e19-4a82-b33b-dea02080290f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"
Sep 08 14:17:02.962108 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.962039    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.962108 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.962066    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqg9z\" (UniqueName: \"kubernetes.io/projected/3fc3293c-9e19-4a82-b33b-dea02080290f-kube-api-access-tqg9z\") pod \"service-ca-operator-859f94c4b-rn2gb\" (UID: \"3fc3293c-9e19-4a82-b33b-dea02080290f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"
Sep 08 14:17:02.962108 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.962092    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxx2l\" (UniqueName: \"kubernetes.io/projected/9e4308cc-2b52-4381-b080-16da804a4de8-kube-api-access-gxx2l\") pod \"network-check-source-685456655f-chktm\" (UID: \"9e4308cc-2b52-4381-b080-16da804a4de8\") " pod="openshift-network-diagnostics/network-check-source-685456655f-chktm"
Sep 08 14:17:02.962380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.962127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3039d4d-4454-4649-8e1a-6dc0f5bfe51b-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cxl6r\" (UID: \"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"
Sep 08 14:17:02.962380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.962167    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fc3293c-9e19-4a82-b33b-dea02080290f-config\") pod \"service-ca-operator-859f94c4b-rn2gb\" (UID: \"3fc3293c-9e19-4a82-b33b-dea02080290f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"
Sep 08 14:17:02.962380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.962183    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.962380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.962261    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:02.962380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.962287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2st2m\" (UniqueName: \"kubernetes.io/projected/cbe9ccca-1669-4132-ada4-d46a8720c7a1-kube-api-access-2st2m\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:02.962380 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:02.962337    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:03.462314714 +0000 UTC m=+38.693245908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : configmap references non-existent config key: service-ca.crt
Sep 08 14:17:02.962673 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:02.962408    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:17:02.962673 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:02.962469    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:03.462456529 +0000 UTC m=+38.693387721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : secret "router-metrics-certs-default" not found
Sep 08 14:17:02.962928 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.962904    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3039d4d-4454-4649-8e1a-6dc0f5bfe51b-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cxl6r\" (UID: \"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"
Sep 08 14:17:02.965739 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.965714    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3039d4d-4454-4649-8e1a-6dc0f5bfe51b-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cxl6r\" (UID: \"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"
Sep 08 14:17:02.965903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.965882    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-default-certificate\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.965972 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.965914    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-stats-auth\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:02.973282 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.973262    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cltqh\" (UniqueName: \"kubernetes.io/projected/d3039d4d-4454-4649-8e1a-6dc0f5bfe51b-kube-api-access-cltqh\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cxl6r\" (UID: \"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"
Sep 08 14:17:02.973782 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:02.973759    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb526\" (UniqueName: \"kubernetes.io/projected/595b339a-11fd-45dc-a950-63dcd5236b02-kube-api-access-rb526\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:03.063578 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.063499    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fc3293c-9e19-4a82-b33b-dea02080290f-serving-cert\") pod \"service-ca-operator-859f94c4b-rn2gb\" (UID: \"3fc3293c-9e19-4a82-b33b-dea02080290f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"
Sep 08 14:17:03.063578 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.063546    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqg9z\" (UniqueName: \"kubernetes.io/projected/3fc3293c-9e19-4a82-b33b-dea02080290f-kube-api-access-tqg9z\") pod \"service-ca-operator-859f94c4b-rn2gb\" (UID: \"3fc3293c-9e19-4a82-b33b-dea02080290f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"
Sep 08 14:17:03.063578 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.063574    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxx2l\" (UniqueName: \"kubernetes.io/projected/9e4308cc-2b52-4381-b080-16da804a4de8-kube-api-access-gxx2l\") pod \"network-check-source-685456655f-chktm\" (UID: \"9e4308cc-2b52-4381-b080-16da804a4de8\") " pod="openshift-network-diagnostics/network-check-source-685456655f-chktm"
Sep 08 14:17:03.063831 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.063623    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fc3293c-9e19-4a82-b33b-dea02080290f-config\") pod \"service-ca-operator-859f94c4b-rn2gb\" (UID: \"3fc3293c-9e19-4a82-b33b-dea02080290f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"
Sep 08 14:17:03.063831 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.063661    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:03.063831 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.063738    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 14:17:03.063831 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.063799    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls podName:cbe9ccca-1669-4132-ada4-d46a8720c7a1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:03.563779024 +0000 UTC m=+38.794710219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-fwr6b" (UID: "cbe9ccca-1669-4132-ada4-d46a8720c7a1") : secret "samples-operator-tls" not found
Sep 08 14:17:03.064028 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.063839    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2st2m\" (UniqueName: \"kubernetes.io/projected/cbe9ccca-1669-4132-ada4-d46a8720c7a1-kube-api-access-2st2m\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:03.064228 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.064188    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fc3293c-9e19-4a82-b33b-dea02080290f-config\") pod \"service-ca-operator-859f94c4b-rn2gb\" (UID: \"3fc3293c-9e19-4a82-b33b-dea02080290f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"
Sep 08 14:17:03.066051 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.066020    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fc3293c-9e19-4a82-b33b-dea02080290f-serving-cert\") pod \"service-ca-operator-859f94c4b-rn2gb\" (UID: \"3fc3293c-9e19-4a82-b33b-dea02080290f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"
Sep 08 14:17:03.076474 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.076453    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2st2m\" (UniqueName: \"kubernetes.io/projected/cbe9ccca-1669-4132-ada4-d46a8720c7a1-kube-api-access-2st2m\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:03.076595 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.076475    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxx2l\" (UniqueName: \"kubernetes.io/projected/9e4308cc-2b52-4381-b080-16da804a4de8-kube-api-access-gxx2l\") pod \"network-check-source-685456655f-chktm\" (UID: \"9e4308cc-2b52-4381-b080-16da804a4de8\") " pod="openshift-network-diagnostics/network-check-source-685456655f-chktm"
Sep 08 14:17:03.076814 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.076794    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqg9z\" (UniqueName: \"kubernetes.io/projected/3fc3293c-9e19-4a82-b33b-dea02080290f-kube-api-access-tqg9z\") pod \"service-ca-operator-859f94c4b-rn2gb\" (UID: \"3fc3293c-9e19-4a82-b33b-dea02080290f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"
Sep 08 14:17:03.086258 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.086235    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"
Sep 08 14:17:03.096983 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.096954    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:03.185927 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.185897    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"
Sep 08 14:17:03.207222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.207184    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-chktm"
Sep 08 14:17:03.371492 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.371458    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r"]
Sep 08 14:17:03.375600 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:03.375566    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3039d4d_4454_4649_8e1a_6dc0f5bfe51b.slice/crio-2acaa0b4f6590256d4ddd2a67f3c99389d33e00570255b94c7252207827292d3 WatchSource:0}: Error finding container 2acaa0b4f6590256d4ddd2a67f3c99389d33e00570255b94c7252207827292d3: Status 404 returned error can't find the container with id 2acaa0b4f6590256d4ddd2a67f3c99389d33e00570255b94c7252207827292d3
Sep 08 14:17:03.445716 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.445673    2558 generic.go:356] "Generic (PLEG): container finished" podID="0f419cc2-7833-4240-8628-3a0356e4f630" containerID="bc4ffd1bd29bf712957503c7f5b9879893c3e418056c34b0768e4db6f1790b9a" exitCode=0
Sep 08 14:17:03.445864 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.445764    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cqkch" event={"ID":"0f419cc2-7833-4240-8628-3a0356e4f630","Type":"ContainerDied","Data":"bc4ffd1bd29bf712957503c7f5b9879893c3e418056c34b0768e4db6f1790b9a"}
Sep 08 14:17:03.447412 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.447377    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r" event={"ID":"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b","Type":"ContainerStarted","Data":"2acaa0b4f6590256d4ddd2a67f3c99389d33e00570255b94c7252207827292d3"}
Sep 08 14:17:03.467969 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.467902    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:03.468076 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.467981    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:03.468076 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.468059    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:17:03.468273 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.468122    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:17:03.468273 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.468174    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:04.468152584 +0000 UTC m=+39.699083776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : configmap references non-existent config key: service-ca.crt
Sep 08 14:17:03.468369 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.468270    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:17:03.468369 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.468337    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls podName:ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:07.468321555 +0000 UTC m=+42.699252747 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls") pod "dns-default-zz6lc" (UID: "ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1") : secret "dns-default-metrics-tls" not found
Sep 08 14:17:03.468462 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.468404    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:17:03.468462 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.468440    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:04.468428222 +0000 UTC m=+39.699359417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : secret "router-metrics-certs-default" not found
Sep 08 14:17:03.468547 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.468503    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 14:17:03.468547 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.468516    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854976b669-4fdlh: secret "image-registry-tls" not found
Sep 08 14:17:03.468644 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.468554    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls podName:8f194f86-4302-46b1-b765-094007b9b1b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:07.468543068 +0000 UTC m=+42.699474259 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls") pod "image-registry-854976b669-4fdlh" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4") : secret "image-registry-tls" not found
Sep 08 14:17:03.480419 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.480393    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb"]
Sep 08 14:17:03.488035 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:03.487577    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc3293c_9e19_4a82_b33b_dea02080290f.slice/crio-0863cd94d0705843224a99a00da3115f2ea23818b3a6dbddaa19407a2029ab9b WatchSource:0}: Error finding container 0863cd94d0705843224a99a00da3115f2ea23818b3a6dbddaa19407a2029ab9b: Status 404 returned error can't find the container with id 0863cd94d0705843224a99a00da3115f2ea23818b3a6dbddaa19407a2029ab9b
Sep 08 14:17:03.499986 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.499958    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-chktm"]
Sep 08 14:17:03.505861 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:03.505824    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e4308cc_2b52_4381_b080_16da804a4de8.slice/crio-57c183c69993b221eb6189ad52801a66e7314c9224fdf4c8154600e516f03ff4 WatchSource:0}: Error finding container 57c183c69993b221eb6189ad52801a66e7314c9224fdf4c8154600e516f03ff4: Status 404 returned error can't find the container with id 57c183c69993b221eb6189ad52801a66e7314c9224fdf4c8154600e516f03ff4
Sep 08 14:17:03.568805 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:03.568776    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:03.568953 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.568929    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 14:17:03.569024 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:03.569013    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls podName:cbe9ccca-1669-4132-ada4-d46a8720c7a1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:04.568992702 +0000 UTC m=+39.799923891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-fwr6b" (UID: "cbe9ccca-1669-4132-ada4-d46a8720c7a1") : secret "samples-operator-tls" not found
Sep 08 14:17:04.453669 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:04.453629    2558 generic.go:356] "Generic (PLEG): container finished" podID="0f419cc2-7833-4240-8628-3a0356e4f630" containerID="d5e8c73b67efe518bc79444888454eb33ff34365551882897cdf0013572be0b2" exitCode=0
Sep 08 14:17:04.454110 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:04.453703    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cqkch" event={"ID":"0f419cc2-7833-4240-8628-3a0356e4f630","Type":"ContainerDied","Data":"d5e8c73b67efe518bc79444888454eb33ff34365551882897cdf0013572be0b2"}
Sep 08 14:17:04.467894 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:04.467868    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-chktm" event={"ID":"9e4308cc-2b52-4381-b080-16da804a4de8","Type":"ContainerStarted","Data":"57c183c69993b221eb6189ad52801a66e7314c9224fdf4c8154600e516f03ff4"}
Sep 08 14:17:04.468940 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:04.468920    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb" event={"ID":"3fc3293c-9e19-4a82-b33b-dea02080290f","Type":"ContainerStarted","Data":"0863cd94d0705843224a99a00da3115f2ea23818b3a6dbddaa19407a2029ab9b"}
Sep 08 14:17:04.477671 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:04.477651    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:04.477744 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:04.477689    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:04.477795 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:04.477788    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:17:04.477829 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:04.477802    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:06.477786496 +0000 UTC m=+41.708717671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : configmap references non-existent config key: service-ca.crt
Sep 08 14:17:04.477829 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:04.477825    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:06.477814248 +0000 UTC m=+41.708745438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : secret "router-metrics-certs-default" not found
Sep 08 14:17:04.578951 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:04.578757    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:04.579052 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:04.578902    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 14:17:04.579122 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:04.579099    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls podName:cbe9ccca-1669-4132-ada4-d46a8720c7a1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:06.579073251 +0000 UTC m=+41.810004448 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-fwr6b" (UID: "cbe9ccca-1669-4132-ada4-d46a8720c7a1") : secret "samples-operator-tls" not found
Sep 08 14:17:04.769531 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:04.769494    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:04.870397 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:04.870362    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:05.267731 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:05.267651    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:05.474507 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:05.474460    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gxn5d" event={"ID":"a1d1aacf-d0b7-4da2-bd12-beaefcc94966","Type":"ContainerStarted","Data":"0b0b309be61e2bd362940e72e15cf2e7ab0e082a4eee06865198eb14789ea149"}
Sep 08 14:17:05.480600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:05.480316    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cqkch" event={"ID":"0f419cc2-7833-4240-8628-3a0356e4f630","Type":"ContainerStarted","Data":"94543350393d919c9b14d83b519c3c3a842d751f5354f767c4b8d82fdc5d5545"}
Sep 08 14:17:05.514824 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:05.514315    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cqkch" podStartSLOduration=8.850954843 podStartE2EDuration="39.514294319s" totalImagesPullingTime="30.663339476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:16:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:05.514142448 +0000 UTC m=+40.745073645" watchObservedRunningTime="2026-09-08 14:17:05.514294319 +0000 UTC m=+40.745225517"
Sep 08 14:17:05.515172 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:05.515048    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gxn5d" podStartSLOduration=2.178709408 podStartE2EDuration="6.515034696s" totalImagesPullingTime="4.336325288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:05.491783575 +0000 UTC m=+40.722714773" watchObservedRunningTime="2026-09-08 14:17:05.515034696 +0000 UTC m=+40.745965894"
Sep 08 14:17:05.647057 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:05.646979    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:05.750725 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:05.750654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:06.495891 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:06.495852    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:06.496411 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:06.496004    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:17:06.496411 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:06.496082    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:10.496062096 +0000 UTC m=+45.726993276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : secret "router-metrics-certs-default" not found
Sep 08 14:17:06.496411 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:06.496154    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:06.496411 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:06.496307    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:10.496294037 +0000 UTC m=+45.727225212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : configmap references non-existent config key: service-ca.crt
Sep 08 14:17:06.597409 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:06.597277    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:06.597409 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:06.597406    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 14:17:06.597618 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:06.597478    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls podName:cbe9ccca-1669-4132-ada4-d46a8720c7a1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:10.597458168 +0000 UTC m=+45.828389371 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-fwr6b" (UID: "cbe9ccca-1669-4132-ada4-d46a8720c7a1") : secret "samples-operator-tls" not found
Sep 08 14:17:06.691349 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:06.691315    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:06.792328 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:06.792262    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:06.941860 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:06.941828    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:07.004890 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:07.004835    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:07.504688 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:07.504602    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:17:07.505096 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:07.504763    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:17:07.505096 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:07.504773    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:17:07.505096 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:07.504828    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls podName:ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:15.504809289 +0000 UTC m=+50.735740483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls") pod "dns-default-zz6lc" (UID: "ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1") : secret "dns-default-metrics-tls" not found
Sep 08 14:17:07.505096 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:07.504892    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 14:17:07.505096 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:07.504906    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854976b669-4fdlh: secret "image-registry-tls" not found
Sep 08 14:17:07.505096 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:07.504959    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls podName:8f194f86-4302-46b1-b765-094007b9b1b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:15.504944613 +0000 UTC m=+50.735875801 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls") pod "image-registry-854976b669-4fdlh" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4") : secret "image-registry-tls" not found
Sep 08 14:17:09.492466 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:09.492423    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb" event={"ID":"3fc3293c-9e19-4a82-b33b-dea02080290f","Type":"ContainerStarted","Data":"63771aec6afe05efc1b8f330eb4458f48cbc1f1cf9d3431576dc45792f5c8a22"}
Sep 08 14:17:09.494236 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:09.494190    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r" event={"ID":"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b","Type":"ContainerStarted","Data":"bd27e5ae75fc4a599fb71d5cdad03fc4cd67c6f04d10c478acbe2abb6c112164"}
Sep 08 14:17:09.495824 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:09.495801    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-chktm" event={"ID":"9e4308cc-2b52-4381-b080-16da804a4de8","Type":"ContainerStarted","Data":"478399a280957b64330a27b0247740aeb0fca4cd0af05d33e01876fedde79042"}
Sep 08 14:17:09.511489 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:09.511432    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb" podStartSLOduration=2.128761111 podStartE2EDuration="7.511415082s" totalImagesPullingTime="5.382653971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:09.510788149 +0000 UTC m=+44.741719347" watchObservedRunningTime="2026-09-08 14:17:09.511415082 +0000 UTC m=+44.742346279"
Sep 08 14:17:09.537754 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:09.537708    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r" podStartSLOduration=2.043711191 podStartE2EDuration="7.537693316s" totalImagesPullingTime="5.493982125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:09.536705374 +0000 UTC m=+44.767636572" watchObservedRunningTime="2026-09-08 14:17:09.537693316 +0000 UTC m=+44.768624513"
Sep 08 14:17:09.557333 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:09.557278    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-chktm" podStartSLOduration=2.186069627 podStartE2EDuration="7.557259069s" totalImagesPullingTime="5.371189442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:09.556577984 +0000 UTC m=+44.787509183" watchObservedRunningTime="2026-09-08 14:17:09.557259069 +0000 UTC m=+44.788190267"
Sep 08 14:17:09.722287 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:09.722251    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:10.368355 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.368321    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-z6jv7"]
Sep 08 14:17:10.386003 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.385968    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z6jv7"]
Sep 08 14:17:10.386146 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.386110    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.388672 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.388648    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 14:17:10.388809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.388722    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 14:17:10.389014 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.388996    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2wblk\""
Sep 08 14:17:10.389215 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.389180    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 14:17:10.389508 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.389364    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 14:17:10.389809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.389783    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:10.455706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.455673    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:10.533770 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.533732    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.534299 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.533788    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:10.534299 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.533841    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98688ac0-dc18-43ff-a88e-83560f34c188-data-volume\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.534299 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.533867    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.534299 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.533883    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r86vc\" (UniqueName: \"kubernetes.io/projected/98688ac0-dc18-43ff-a88e-83560f34c188-kube-api-access-r86vc\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.534299 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.533950    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:10.534299 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:10.534004    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:17:10.534299 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:10.534055    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:18.534037465 +0000 UTC m=+53.764968643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : configmap references non-existent config key: service-ca.crt
Sep 08 14:17:10.534299 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:10.534077    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:18.534068104 +0000 UTC m=+53.764999283 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : secret "router-metrics-certs-default" not found
Sep 08 14:17:10.534299 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.534101    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98688ac0-dc18-43ff-a88e-83560f34c188-crio-socket\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.635414 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.635320    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98688ac0-dc18-43ff-a88e-83560f34c188-data-volume\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.635414 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.635380    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.635414 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.635407    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r86vc\" (UniqueName: \"kubernetes.io/projected/98688ac0-dc18-43ff-a88e-83560f34c188-kube-api-access-r86vc\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.635722 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.635456    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:10.635722 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:10.635483    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 14:17:10.635722 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:10.635562    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm podName:98688ac0-dc18-43ff-a88e-83560f34c188 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:11.135543398 +0000 UTC m=+46.366474575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z6jv7" (UID: "98688ac0-dc18-43ff-a88e-83560f34c188") : configmap "kube-rbac-proxy" not found
Sep 08 14:17:10.635722 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.635619    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98688ac0-dc18-43ff-a88e-83560f34c188-crio-socket\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.635722 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.635717    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.635977 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.635723    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98688ac0-dc18-43ff-a88e-83560f34c188-data-volume\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.635977 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:10.635874    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 14:17:10.635977 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.635880    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98688ac0-dc18-43ff-a88e-83560f34c188-crio-socket\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.635977 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:10.635919    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls podName:98688ac0-dc18-43ff-a88e-83560f34c188 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:11.135905278 +0000 UTC m=+46.366836459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z6jv7" (UID: "98688ac0-dc18-43ff-a88e-83560f34c188") : secret "insights-runtime-extractor-tls" not found
Sep 08 14:17:10.636167 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:10.636004    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 14:17:10.636167 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:10.636044    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls podName:cbe9ccca-1669-4132-ada4-d46a8720c7a1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:18.636031744 +0000 UTC m=+53.866962922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-fwr6b" (UID: "cbe9ccca-1669-4132-ada4-d46a8720c7a1") : secret "samples-operator-tls" not found
Sep 08 14:17:10.647870 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.647842    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r86vc\" (UniqueName: \"kubernetes.io/projected/98688ac0-dc18-43ff-a88e-83560f34c188-kube-api-access-r86vc\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:10.840045 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.840009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:10.903250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:10.903104    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:11.140128 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:11.140088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:11.140328 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:11.140234    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:11.140418 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:11.140397    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 14:17:11.140550 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:11.140475    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm podName:98688ac0-dc18-43ff-a88e-83560f34c188 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:12.1404564 +0000 UTC m=+47.371387579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z6jv7" (UID: "98688ac0-dc18-43ff-a88e-83560f34c188") : configmap "kube-rbac-proxy" not found
Sep 08 14:17:11.140550 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:11.140472    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 14:17:11.140550 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:11.140512    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls podName:98688ac0-dc18-43ff-a88e-83560f34c188 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:12.14050255 +0000 UTC m=+47.371433730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z6jv7" (UID: "98688ac0-dc18-43ff-a88e-83560f34c188") : secret "insights-runtime-extractor-tls" not found
Sep 08 14:17:11.206635 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:11.206554    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:11.294808 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:11.294775    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:11.358696 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:11.358662    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:11.547965 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:11.547932    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:11.608689 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:11.608654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:11.946662 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:11.946597    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-shfmt_b23c51e8-9fb2-4e5c-883a-65b643ed3ef2/dns-node-resolver/0.log"
Sep 08 14:17:12.021726 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:12.021691    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:12.082368 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:12.082326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:12.149696 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:12.149659    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:12.149874 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:12.149764    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:12.149874 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:12.149814    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 14:17:12.149982 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:12.149874    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 14:17:12.149982 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:12.149892    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm podName:98688ac0-dc18-43ff-a88e-83560f34c188 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:14.149871537 +0000 UTC m=+49.380802712 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z6jv7" (UID: "98688ac0-dc18-43ff-a88e-83560f34c188") : configmap "kube-rbac-proxy" not found
Sep 08 14:17:12.149982 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:12.149916    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls podName:98688ac0-dc18-43ff-a88e-83560f34c188 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:14.14990498 +0000 UTC m=+49.380836155 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z6jv7" (UID: "98688ac0-dc18-43ff-a88e-83560f34c188") : secret "insights-runtime-extractor-tls" not found
Sep 08 14:17:12.946786 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:12.946761    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7c4rl_859d3474-55bb-4163-b637-cf8a13e219d4/node-ca/0.log"
Sep 08 14:17:13.202872 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:13.202802    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:13.266141 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:13.266101    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:13.545663 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:13.545632    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gxn5d_a1d1aacf-d0b7-4da2-bd12-beaefcc94966/serve-healthcheck-canary/0.log"
Sep 08 14:17:14.069663 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:14.069629    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:14.167873 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:14.167829    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:14.168038 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:14.167960    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 14:17:14.168038 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:14.168000    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:14.168038 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:14.168027    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm podName:98688ac0-dc18-43ff-a88e-83560f34c188 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:18.168009257 +0000 UTC m=+53.398940444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z6jv7" (UID: "98688ac0-dc18-43ff-a88e-83560f34c188") : configmap "kube-rbac-proxy" not found
Sep 08 14:17:14.168153 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:14.168075    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 14:17:14.168153 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:14.168145    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls podName:98688ac0-dc18-43ff-a88e-83560f34c188 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:18.168133274 +0000 UTC m=+53.399064448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z6jv7" (UID: "98688ac0-dc18-43ff-a88e-83560f34c188") : secret "insights-runtime-extractor-tls" not found
Sep 08 14:17:15.579267 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:15.579201    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:17:15.579750 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:15.579297    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:17:15.579750 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:15.579335    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:17:15.579750 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:15.579396    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 14:17:15.579750 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:15.579413    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854976b669-4fdlh: secret "image-registry-tls" not found
Sep 08 14:17:15.579750 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:15.579435    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls podName:ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:31.579410502 +0000 UTC m=+66.810341679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls") pod "dns-default-zz6lc" (UID: "ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1") : secret "dns-default-metrics-tls" not found
Sep 08 14:17:15.579750 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:15.579457    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls podName:8f194f86-4302-46b1-b765-094007b9b1b4 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:31.579445832 +0000 UTC m=+66.810377006 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls") pod "image-registry-854976b669-4fdlh" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4") : secret "image-registry-tls" not found
Sep 08 14:17:15.736474 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:15.736438    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:15.804354 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:15.804304    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:16.068575 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:16.068541    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:17.632016 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:17.631978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:17.703636 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:17.703603    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:17.883439 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:17.883348    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:18.201715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:18.201621    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:18.201870 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:18.201724    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:18.201870 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:18.201779    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 14:17:18.201870 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:18.201860    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls podName:98688ac0-dc18-43ff-a88e-83560f34c188 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:26.201840434 +0000 UTC m=+61.432771614 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z6jv7" (UID: "98688ac0-dc18-43ff-a88e-83560f34c188") : secret "insights-runtime-extractor-tls" not found
Sep 08 14:17:18.202353 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:18.202336    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98688ac0-dc18-43ff-a88e-83560f34c188-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:18.605468 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:18.605415    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:18.605678 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:18.605506    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:18.605678 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:18.605581    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:34.605564664 +0000 UTC m=+69.836495839 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : configmap references non-existent config key: service-ca.crt
Sep 08 14:17:18.605678 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:18.605641    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:17:18.605790 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:18.605695    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs podName:595b339a-11fd-45dc-a950-63dcd5236b02 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:34.605683218 +0000 UTC m=+69.836614397 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs") pod "router-default-585d697d6-dm2q6" (UID: "595b339a-11fd-45dc-a950-63dcd5236b02") : secret "router-metrics-certs-default" not found
Sep 08 14:17:18.706454 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:18.706407    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:18.706822 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:18.706545    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 14:17:18.706822 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:17:18.706605    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls podName:cbe9ccca-1669-4132-ada4-d46a8720c7a1 nodeName:}" failed. No retries permitted until 2026-09-08 14:17:34.70659034 +0000 UTC m=+69.937521515 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-fwr6b" (UID: "cbe9ccca-1669-4132-ada4-d46a8720c7a1") : secret "samples-operator-tls" not found
Sep 08 14:17:18.899700 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:18.899619    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:19.129445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:19.129417    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:19.239849 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:19.239821    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:19.340447 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:19.340411    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:19.409841 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:19.409805    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:19.603609 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:19.603521    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:20.599625 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:20.599591    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:21.002915 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:21.002888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:21.194049 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:21.194009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:21.853901 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:21.853870    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:21.911918 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:21.911883    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:22.398754 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:22.398721    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:22.595891 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:22.595858    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:22.802519 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:22.802488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:23.146798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:23.146718    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:23.248490 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:23.248457    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:23.596476 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:23.596444    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:23.798495 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:23.798469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:23.900428 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:23.900360    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:24.399077 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:24.399048    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:25.009914 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:25.009875    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:25.192932 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:25.192901    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:25.293823 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:25.293758    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:25.355383 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:25.355351    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:25.414499 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:25.414461    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:25.802406 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:25.802371    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:26.270151 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:26.270119    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:26.272492 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:26.272467    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98688ac0-dc18-43ff-a88e-83560f34c188-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z6jv7\" (UID: \"98688ac0-dc18-43ff-a88e-83560f34c188\") " pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:26.300350 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:26.300324    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2wblk\""
Sep 08 14:17:26.308569 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:26.308552    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z6jv7"
Sep 08 14:17:26.431286 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:26.431261    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qb4z"
Sep 08 14:17:26.586110 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:26.586040    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z6jv7"]
Sep 08 14:17:26.589024 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:26.589000    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98688ac0_dc18_43ff_a88e_83560f34c188.slice/crio-b1a646cbc4a74b472cfef6ef592bcec027c90e8e7e368dfe59e0aa09128584a4 WatchSource:0}: Error finding container b1a646cbc4a74b472cfef6ef592bcec027c90e8e7e368dfe59e0aa09128584a4: Status 404 returned error can't find the container with id b1a646cbc4a74b472cfef6ef592bcec027c90e8e7e368dfe59e0aa09128584a4
Sep 08 14:17:26.611178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:26.611144    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:26.998488 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:26.998456    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:27.223598 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:27.223564    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:27.542110 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:27.542081    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z6jv7" event={"ID":"98688ac0-dc18-43ff-a88e-83560f34c188","Type":"ContainerStarted","Data":"4c42c880b8de5740a1ba765a70a8eb2a8f69fe6b61efc2c867659611fb2249a1"}
Sep 08 14:17:27.542409 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:27.542143    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z6jv7" event={"ID":"98688ac0-dc18-43ff-a88e-83560f34c188","Type":"ContainerStarted","Data":"b1a646cbc4a74b472cfef6ef592bcec027c90e8e7e368dfe59e0aa09128584a4"}
Sep 08 14:17:27.894220 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:27.894129    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:27.954591 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:27.954544    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:28.547763 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:28.547728    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z6jv7" event={"ID":"98688ac0-dc18-43ff-a88e-83560f34c188","Type":"ContainerStarted","Data":"b85fa197784ae172987170d743d77a06fd799582dc649f17a01da31c50261af4"}
Sep 08 14:17:28.801833 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:28.801760    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:29.004793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:29.004749    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:29.197740 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:29.197707    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:29.411060 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:29.410979    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:29.552365 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:29.552332    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z6jv7" event={"ID":"98688ac0-dc18-43ff-a88e-83560f34c188","Type":"ContainerStarted","Data":"40b65d369e0f816e04d088255fe9eb67897ba1312cc5de2ba932f7b8bce8565a"}
Sep 08 14:17:29.575149 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:29.575101    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-z6jv7" podStartSLOduration=17.170314281 podStartE2EDuration="19.575086878s" totalImagesPullingTime="2.404772597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:17:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:29.574337971 +0000 UTC m=+64.805269168" watchObservedRunningTime="2026-09-08 14:17:29.575086878 +0000 UTC m=+64.806018112"
Sep 08 14:17:30.118266 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:30.118230    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:30.217609 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:30.217570    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:30.492134 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:30.492104    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:31.009329 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.009296    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:17:31.011818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.011800    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 14:17:31.022290 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.022269    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96b871d-efd7-43e9-b268-978c310e94b4-metrics-certs\") pod \"network-metrics-daemon-zcpxc\" (UID: \"d96b871d-efd7-43e9-b268-978c310e94b4\") " pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:17:31.109788 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.109742    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttwg\" (UniqueName: \"kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg\") pod \"network-check-target-92v85\" (UID: \"237de209-b858-44fd-ae88-a4ec84cf0d37\") " pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:17:31.109927 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.109823    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:17:31.112301 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.112269    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ttwg\" (UniqueName: \"kubernetes.io/projected/237de209-b858-44fd-ae88-a4ec84cf0d37-kube-api-access-8ttwg\") pod \"network-check-target-92v85\" (UID: \"237de209-b858-44fd-ae88-a4ec84cf0d37\") " pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:17:31.112569 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.112552    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 14:17:31.122509 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.122489    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e20eb2e-4a15-409a-9192-aafeec619b23-original-pull-secret\") pod \"global-pull-secret-syncer-drbnw\" (UID: \"4e20eb2e-4a15-409a-9192-aafeec619b23\") " pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:17:31.194438 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.194397    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-drbnw"
Sep 08 14:17:31.204946 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.204926    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vvt7d\""
Sep 08 14:17:31.211638 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.211618    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zbnn7\""
Sep 08 14:17:31.212603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.212588    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:17:31.219457 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.219437    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcpxc"
Sep 08 14:17:31.464068 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.464037    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-drbnw"]
Sep 08 14:17:31.466645 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:31.466620    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e20eb2e_4a15_409a_9192_aafeec619b23.slice/crio-42888558515e7db282527cf3035515226c0e4daa9070536f692240be1c73475d WatchSource:0}: Error finding container 42888558515e7db282527cf3035515226c0e4daa9070536f692240be1c73475d: Status 404 returned error can't find the container with id 42888558515e7db282527cf3035515226c0e4daa9070536f692240be1c73475d
Sep 08 14:17:31.489681 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.489657    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-92v85"]
Sep 08 14:17:31.492049 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:31.492027    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237de209_b858_44fd_ae88_a4ec84cf0d37.slice/crio-5594e3bf6a739433492f536e65275a32e155140102d68b34f25a1f958bd4cc21 WatchSource:0}: Error finding container 5594e3bf6a739433492f536e65275a32e155140102d68b34f25a1f958bd4cc21: Status 404 returned error can't find the container with id 5594e3bf6a739433492f536e65275a32e155140102d68b34f25a1f958bd4cc21
Sep 08 14:17:31.514201 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.514179    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zcpxc"]
Sep 08 14:17:31.517253 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:31.517157    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd96b871d_efd7_43e9_b268_978c310e94b4.slice/crio-1e6e4c277c02ee09ce566b13b9e232548cfb23c559c1c382cabe4390b822c946 WatchSource:0}: Error finding container 1e6e4c277c02ee09ce566b13b9e232548cfb23c559c1c382cabe4390b822c946: Status 404 returned error can't find the container with id 1e6e4c277c02ee09ce566b13b9e232548cfb23c559c1c382cabe4390b822c946
Sep 08 14:17:31.559716 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.559689    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zcpxc" event={"ID":"d96b871d-efd7-43e9-b268-978c310e94b4","Type":"ContainerStarted","Data":"1e6e4c277c02ee09ce566b13b9e232548cfb23c559c1c382cabe4390b822c946"}
Sep 08 14:17:31.561082 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.561062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-92v85" event={"ID":"237de209-b858-44fd-ae88-a4ec84cf0d37","Type":"ContainerStarted","Data":"c026f498c99bb1b6afe137f4cb82b338459913f2e0682584e0552d25185fd6c0"}
Sep 08 14:17:31.561187 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.561101    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-92v85" event={"ID":"237de209-b858-44fd-ae88-a4ec84cf0d37","Type":"ContainerStarted","Data":"5594e3bf6a739433492f536e65275a32e155140102d68b34f25a1f958bd4cc21"}
Sep 08 14:17:31.561270 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.561241    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:17:31.562178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.562157    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-drbnw" event={"ID":"4e20eb2e-4a15-409a-9192-aafeec619b23","Type":"ContainerStarted","Data":"42888558515e7db282527cf3035515226c0e4daa9070536f692240be1c73475d"}
Sep 08 14:17:31.579080 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.579038    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-92v85" podStartSLOduration=66.579025145 podStartE2EDuration="1m6.579025145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:16:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:31.578381371 +0000 UTC m=+66.809312569" watchObservedRunningTime="2026-09-08 14:17:31.579025145 +0000 UTC m=+66.809956343"
Sep 08 14:17:31.612910 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.612891    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:17:31.613099 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.613079    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:17:31.615119 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.615094    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1-metrics-tls\") pod \"dns-default-zz6lc\" (UID: \"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1\") " pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:17:31.615268 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.615253    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls\") pod \"image-registry-854976b669-4fdlh\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") " pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:17:31.781189 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.781092    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:31.812699 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.812674    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kj4kg\""
Sep 08 14:17:31.820631 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.820609    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:17:31.825754 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.825738    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rlxm6\""
Sep 08 14:17:31.832876 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.832857    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:17:31.881989 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.881893    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:31.982400 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:31.982374    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:32.097649 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:32.097575    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-854976b669-4fdlh"]
Sep 08 14:17:32.101670 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:32.101639    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f194f86_4302_46b1_b765_094007b9b1b4.slice/crio-7de6a3bfbacb8ff25c6d2750b2c634ae5a6cfb5344a9677929544eb66045b4d6 WatchSource:0}: Error finding container 7de6a3bfbacb8ff25c6d2750b2c634ae5a6cfb5344a9677929544eb66045b4d6: Status 404 returned error can't find the container with id 7de6a3bfbacb8ff25c6d2750b2c634ae5a6cfb5344a9677929544eb66045b4d6
Sep 08 14:17:32.131618 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:32.131545    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zz6lc"]
Sep 08 14:17:32.142669 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:32.142639    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc6dc26_8c51_4dac_93a2_8b5a2924c5d1.slice/crio-3601d6c54051ebf340cf0e41a5f7f028761d25bf7b7d4939e9070ba353866732 WatchSource:0}: Error finding container 3601d6c54051ebf340cf0e41a5f7f028761d25bf7b7d4939e9070ba353866732: Status 404 returned error can't find the container with id 3601d6c54051ebf340cf0e41a5f7f028761d25bf7b7d4939e9070ba353866732
Sep 08 14:17:32.203673 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:32.203643    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:32.567888 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:32.567852    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zz6lc" event={"ID":"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1","Type":"ContainerStarted","Data":"3601d6c54051ebf340cf0e41a5f7f028761d25bf7b7d4939e9070ba353866732"}
Sep 08 14:17:32.569799 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:32.569770    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-854976b669-4fdlh" event={"ID":"8f194f86-4302-46b1-b765-094007b9b1b4","Type":"ContainerStarted","Data":"d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048"}
Sep 08 14:17:32.569922 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:32.569819    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-854976b669-4fdlh" event={"ID":"8f194f86-4302-46b1-b765-094007b9b1b4","Type":"ContainerStarted","Data":"7de6a3bfbacb8ff25c6d2750b2c634ae5a6cfb5344a9677929544eb66045b4d6"}
Sep 08 14:17:32.592126 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:32.592078    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-854976b669-4fdlh" podStartSLOduration=66.592061203 podStartE2EDuration="1m6.592061203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:32.590242873 +0000 UTC m=+67.821174070" watchObservedRunningTime="2026-09-08 14:17:32.592061203 +0000 UTC m=+67.822992399"
Sep 08 14:17:32.736276 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:32.736171    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:32.836490 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:32.836420    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:33.003081 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.003042    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"]
Sep 08 14:17:33.005567 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.005525    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:33.008038 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.008015    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 08 14:17:33.008442 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.008296    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 08 14:17:33.008550 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.008528    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 08 14:17:33.009729 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.009634    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 08 14:17:33.025962 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.025659    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"]
Sep 08 14:17:33.081283 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.081196    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:33.142535 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.142442    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/88533982-21de-437c-8fbf-517e456390c3-klusterlet-config\") pod \"klusterlet-addon-workmgr-7465d78f87-5bpf9\" (UID: \"88533982-21de-437c-8fbf-517e456390c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:33.142974 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.142562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwwbp\" (UniqueName: \"kubernetes.io/projected/88533982-21de-437c-8fbf-517e456390c3-kube-api-access-xwwbp\") pod \"klusterlet-addon-workmgr-7465d78f87-5bpf9\" (UID: \"88533982-21de-437c-8fbf-517e456390c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:33.142974 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.142649    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88533982-21de-437c-8fbf-517e456390c3-tmp\") pod \"klusterlet-addon-workmgr-7465d78f87-5bpf9\" (UID: \"88533982-21de-437c-8fbf-517e456390c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:33.166901 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.166869    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:33.243634 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.243537    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwwbp\" (UniqueName: \"kubernetes.io/projected/88533982-21de-437c-8fbf-517e456390c3-kube-api-access-xwwbp\") pod \"klusterlet-addon-workmgr-7465d78f87-5bpf9\" (UID: \"88533982-21de-437c-8fbf-517e456390c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:33.243634 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.243587    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88533982-21de-437c-8fbf-517e456390c3-tmp\") pod \"klusterlet-addon-workmgr-7465d78f87-5bpf9\" (UID: \"88533982-21de-437c-8fbf-517e456390c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:33.243887 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.243644    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/88533982-21de-437c-8fbf-517e456390c3-klusterlet-config\") pod \"klusterlet-addon-workmgr-7465d78f87-5bpf9\" (UID: \"88533982-21de-437c-8fbf-517e456390c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:33.244107 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.244057    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88533982-21de-437c-8fbf-517e456390c3-tmp\") pod \"klusterlet-addon-workmgr-7465d78f87-5bpf9\" (UID: \"88533982-21de-437c-8fbf-517e456390c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:33.246842 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.246812    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/88533982-21de-437c-8fbf-517e456390c3-klusterlet-config\") pod \"klusterlet-addon-workmgr-7465d78f87-5bpf9\" (UID: \"88533982-21de-437c-8fbf-517e456390c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:33.251707 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.251637    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:33.254723 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.254701    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwwbp\" (UniqueName: \"kubernetes.io/projected/88533982-21de-437c-8fbf-517e456390c3-kube-api-access-xwwbp\") pod \"klusterlet-addon-workmgr-7465d78f87-5bpf9\" (UID: \"88533982-21de-437c-8fbf-517e456390c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:33.318734 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.318698    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:33.575123 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.575084    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zcpxc" event={"ID":"d96b871d-efd7-43e9-b268-978c310e94b4","Type":"ContainerStarted","Data":"4309e3016aa88bf52902d29ee4e5132edf1cd340921e95a5f0bf473c86bcacf3"}
Sep 08 14:17:33.575437 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.575419    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:17:33.575504 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.575460    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zcpxc" event={"ID":"d96b871d-efd7-43e9-b268-978c310e94b4","Type":"ContainerStarted","Data":"a188cfe46faf9b2451b2fe8140424906f6e4319562807cd35550e13f9b10a198"}
Sep 08 14:17:33.606438 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.606325    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zcpxc" podStartSLOduration=66.498094798 podStartE2EDuration="1m7.606301268s" totalImagesPullingTime="1.10820647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:33.594459168 +0000 UTC m=+68.825390367" watchObservedRunningTime="2026-09-08 14:17:33.606301268 +0000 UTC m=+68.837232464"
Sep 08 14:17:33.607721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.607692    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"]
Sep 08 14:17:33.840755 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:33.840642    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:34.063981 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:34.063934    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88533982_21de_437c_8fbf_517e456390c3.slice/crio-d74e67a564e4416c6433677bbd9661bcf6329ccb6c56110b7b8fec9f0fac7821 WatchSource:0}: Error finding container d74e67a564e4416c6433677bbd9661bcf6329ccb6c56110b7b8fec9f0fac7821: Status 404 returned error can't find the container with id d74e67a564e4416c6433677bbd9661bcf6329ccb6c56110b7b8fec9f0fac7821
Sep 08 14:17:34.237782 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.237409    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:34.304792 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.304761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:34.581954 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.581910    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zz6lc" event={"ID":"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1","Type":"ContainerStarted","Data":"0d4e6e28aa1be69244f1d632215d05e569aab246e7422d47db9b8c16185ddce2"}
Sep 08 14:17:34.582137 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.581978    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zz6lc" event={"ID":"ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1","Type":"ContainerStarted","Data":"b7046d879b0964ec3c4dcf705b59d21b3a921bdbdd585efb45f79c081ab75a9b"}
Sep 08 14:17:34.582233 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.582145    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:17:34.583318 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.583287    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9" event={"ID":"88533982-21de-437c-8fbf-517e456390c3","Type":"ContainerStarted","Data":"d74e67a564e4416c6433677bbd9661bcf6329ccb6c56110b7b8fec9f0fac7821"}
Sep 08 14:17:34.602269 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.602222    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zz6lc" podStartSLOduration=33.651812239 podStartE2EDuration="35.602187733s" totalImagesPullingTime="1.950375494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:34.601697815 +0000 UTC m=+69.832629015" watchObservedRunningTime="2026-09-08 14:17:34.602187733 +0000 UTC m=+69.833118930"
Sep 08 14:17:34.656246 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.656193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:34.656424 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.656332    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:34.657018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.656968    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595b339a-11fd-45dc-a950-63dcd5236b02-service-ca-bundle\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:34.659257 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.659234    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/595b339a-11fd-45dc-a950-63dcd5236b02-metrics-certs\") pod \"router-default-585d697d6-dm2q6\" (UID: \"595b339a-11fd-45dc-a950-63dcd5236b02\") " pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:34.757798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.757753    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:34.760509 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.760477    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbe9ccca-1669-4132-ada4-d46a8720c7a1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-fwr6b\" (UID: \"cbe9ccca-1669-4132-ada4-d46a8720c7a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:34.901372 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.901281    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f6vv4\""
Sep 08 14:17:34.909608 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.909372    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:34.996761 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:34.996552    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zvvdc\""
Sep 08 14:17:35.004771 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:35.004563    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"
Sep 08 14:17:35.089552 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:35.089466    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:35.449180 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:35.449110    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:35.522496 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:35.522460    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:39.085018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.084994    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-585d697d6-dm2q6"]
Sep 08 14:17:39.087340 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.087315    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b"]
Sep 08 14:17:39.191461 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:39.191427    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595b339a_11fd_45dc_a950_63dcd5236b02.slice/crio-3971214d4ca32e334fb556a7b0830729e2cdfdaec51d604a73f8e45b93f0a09d WatchSource:0}: Error finding container 3971214d4ca32e334fb556a7b0830729e2cdfdaec51d604a73f8e45b93f0a09d: Status 404 returned error can't find the container with id 3971214d4ca32e334fb556a7b0830729e2cdfdaec51d604a73f8e45b93f0a09d
Sep 08 14:17:39.602346 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.602303    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9" event={"ID":"88533982-21de-437c-8fbf-517e456390c3","Type":"ContainerStarted","Data":"3eb037dac06f5924b879611937f9cce3ebecd4996d9005d2ac59b16f4fa7a053"}
Sep 08 14:17:39.602537 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.602490    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:39.603979 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.603948    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-drbnw" event={"ID":"4e20eb2e-4a15-409a-9192-aafeec619b23","Type":"ContainerStarted","Data":"a0c21f1bb1eb39e5242a61f1d66d137ea343141ed07f30ed5029da1f02553d8b"}
Sep 08 14:17:39.604315 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.604294    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9"
Sep 08 14:17:39.605161 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.605142    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b" event={"ID":"cbe9ccca-1669-4132-ada4-d46a8720c7a1","Type":"ContainerStarted","Data":"9c41e9dec21845666227f6816224406c3663e42c2827d49eeb813c3f1af88a43"}
Sep 08 14:17:39.606608 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.606581    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-585d697d6-dm2q6" event={"ID":"595b339a-11fd-45dc-a950-63dcd5236b02","Type":"ContainerStarted","Data":"681e928f86a9a47db94234ff419b1a8b107049be3a9f5e4e2971d7bf3ea226ca"}
Sep 08 14:17:39.606701 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.606618    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-585d697d6-dm2q6" event={"ID":"595b339a-11fd-45dc-a950-63dcd5236b02","Type":"ContainerStarted","Data":"3971214d4ca32e334fb556a7b0830729e2cdfdaec51d604a73f8e45b93f0a09d"}
Sep 08 14:17:39.620549 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.620511    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7465d78f87-5bpf9" podStartSLOduration=2.452648331 podStartE2EDuration="7.620498508s" totalImagesPullingTime="5.167850177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:39.619574104 +0000 UTC m=+74.850505302" watchObservedRunningTime="2026-09-08 14:17:39.620498508 +0000 UTC m=+74.851429701"
Sep 08 14:17:39.644029 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.643986    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-585d697d6-dm2q6" podStartSLOduration=37.643972533 podStartE2EDuration="37.643972533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:39.642912142 +0000 UTC m=+74.873843340" watchObservedRunningTime="2026-09-08 14:17:39.643972533 +0000 UTC m=+74.874903730"
Sep 08 14:17:39.678600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.678554    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-drbnw" podStartSLOduration=66.326665117 podStartE2EDuration="1m13.678539607s" totalImagesPullingTime="7.35187449s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:39.677831234 +0000 UTC m=+74.908762431" watchObservedRunningTime="2026-09-08 14:17:39.678539607 +0000 UTC m=+74.909470804"
Sep 08 14:17:39.748831 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.748801    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bf8588899-pkqth"]
Sep 08 14:17:39.764490 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.764463    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bf8588899-pkqth"]
Sep 08 14:17:39.764634 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.764553    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:39.767493 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.767159    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 08 14:17:39.767493 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.767168    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 08 14:17:39.767493 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.767337    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kd8vx\""
Sep 08 14:17:39.767493 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.767396    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 08 14:17:39.767493 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.767473    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 08 14:17:39.767894 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.767541    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 08 14:17:39.767894 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.767775    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 08 14:17:39.767894 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.767816    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 08 14:17:39.900727 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.900641    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-oauth-serving-cert\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:39.900727 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.900688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-serving-cert\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:39.900727 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.900712    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-service-ca\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:39.900954 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.900758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-config\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:39.900954 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.900797    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j27sv\" (UniqueName: \"kubernetes.io/projected/a7f1e1d8-0294-4874-9ed3-289d58f5714b-kube-api-access-j27sv\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:39.900954 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.900866    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-oauth-config\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:39.909582 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.909552    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:39.912504 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:39.912482    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:40.001740 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.001698    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-config\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.001940 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.001755    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j27sv\" (UniqueName: \"kubernetes.io/projected/a7f1e1d8-0294-4874-9ed3-289d58f5714b-kube-api-access-j27sv\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.001940 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.001784    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-oauth-config\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.001940 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.001842    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-oauth-serving-cert\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.001940 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.001875    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-serving-cert\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.001940 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.001903    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-service-ca\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.002558 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.002532    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-config\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.002670 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.002569    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-service-ca\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.002710 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.002683    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-oauth-serving-cert\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.004435 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.004414    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-oauth-config\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.004511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.004501    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-serving-cert\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.011594 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.011568    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j27sv\" (UniqueName: \"kubernetes.io/projected/a7f1e1d8-0294-4874-9ed3-289d58f5714b-kube-api-access-j27sv\") pod \"console-7bf8588899-pkqth\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") " pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.074498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.074459    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:40.378568 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.378534    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bf8588899-pkqth"]
Sep 08 14:17:40.381895 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:40.381857    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7f1e1d8_0294_4874_9ed3_289d58f5714b.slice/crio-d768f83817268b2f03f4b829a42f1cadb446db0a9d9cea731398212721eab7df WatchSource:0}: Error finding container d768f83817268b2f03f4b829a42f1cadb446db0a9d9cea731398212721eab7df: Status 404 returned error can't find the container with id d768f83817268b2f03f4b829a42f1cadb446db0a9d9cea731398212721eab7df
Sep 08 14:17:40.450923 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.450841    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:40.570038 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.570005    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:40.612274 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.612228    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf8588899-pkqth" event={"ID":"a7f1e1d8-0294-4874-9ed3-289d58f5714b","Type":"ContainerStarted","Data":"d768f83817268b2f03f4b829a42f1cadb446db0a9d9cea731398212721eab7df"}
Sep 08 14:17:40.612635 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.612612    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:40.613865 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:40.613845    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-585d697d6-dm2q6"
Sep 08 14:17:41.315135 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:41.315104    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:41.416161 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:41.416136    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:41.494567 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:41.494542    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:41.618585 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:41.618550    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b" event={"ID":"cbe9ccca-1669-4132-ada4-d46a8720c7a1","Type":"ContainerStarted","Data":"eeb0c0bc946fe6d3158de5e25fd7c525b6df756ab795c7d66348e34673c3bbc9"}
Sep 08 14:17:41.842025 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:41.841935    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:42.624604 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:42.624557    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b" event={"ID":"cbe9ccca-1669-4132-ada4-d46a8720c7a1","Type":"ContainerStarted","Data":"deb690a6029458f405a5b1ab4fbca0443bf5ae0260696d6458e8390b735a00be"}
Sep 08 14:17:42.642069 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:42.642016    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-fwr6b" podStartSLOduration=38.489655367 podStartE2EDuration="40.641996051s" totalImagesPullingTime="2.152340684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:42.641489444 +0000 UTC m=+77.872420642" watchObservedRunningTime="2026-09-08 14:17:42.641996051 +0000 UTC m=+77.872927252"
Sep 08 14:17:43.252833 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:43.252797    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:43.307427 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:43.307396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:44.589356 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:44.589323    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zz6lc"
Sep 08 14:17:44.633706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:44.633672    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf8588899-pkqth" event={"ID":"a7f1e1d8-0294-4874-9ed3-289d58f5714b","Type":"ContainerStarted","Data":"a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f"}
Sep 08 14:17:44.652031 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:44.651985    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bf8588899-pkqth" podStartSLOduration=2.133291685 podStartE2EDuration="5.651970324s" totalImagesPullingTime="3.518678639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:17:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:44.651201163 +0000 UTC m=+79.882132360" watchObservedRunningTime="2026-09-08 14:17:44.651970324 +0000 UTC m=+79.882901521"
Sep 08 14:17:44.886570 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:44.886487    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:44.946499 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:44.946467    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:45.518574 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:45.518526    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:45.588410 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:45.588373    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:45.735500 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:45.735464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:45.836552 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:45.836456    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:45.936426 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:45.936372    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:46.012436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:46.012407    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:46.086638 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:46.086561    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:48.238891 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:48.238768    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:50.075317 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.075280    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:50.075842 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.075335    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:50.080656 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.080630    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:50.344972 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.344874    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:50.445588 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.445556    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:50.569420 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.569385    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:50.660635 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.660567    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:17:50.671004 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.670977    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:50.772682 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.772646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:50.911643 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.911561    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:50.983749 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.983720    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fknj7"]
Sep 08 14:17:50.986758 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.986739    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:50.989556 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.989529    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 14:17:50.989556 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.989547    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 14:17:50.989694 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.989552    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 14:17:50.989694 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.989610    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 14:17:50.990628 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.990608    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 14:17:50.990720 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.990662    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w6g7w\""
Sep 08 14:17:50.990720 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:50.990678    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 14:17:51.012857 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.012829    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:51.090256 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.090224    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.090718 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.090261    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a907b903-a15e-487c-87ed-e2852618c431-sys\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.090718 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.090366    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgwnn\" (UniqueName: \"kubernetes.io/projected/a907b903-a15e-487c-87ed-e2852618c431-kube-api-access-fgwnn\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.090718 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.090428    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-wtmp\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.090718 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.090465    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a907b903-a15e-487c-87ed-e2852618c431-root\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.090718 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.090496    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-accelerators-collector-config\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.090718 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.090534    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-tls\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.090718 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.090589    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-textfile\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.090718 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.090634    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a907b903-a15e-487c-87ed-e2852618c431-metrics-client-ca\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.110639 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.110607    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:51.191248 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a907b903-a15e-487c-87ed-e2852618c431-sys\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191248 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191183    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgwnn\" (UniqueName: \"kubernetes.io/projected/a907b903-a15e-487c-87ed-e2852618c431-kube-api-access-fgwnn\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191248 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191242    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-wtmp\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191516 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191270    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a907b903-a15e-487c-87ed-e2852618c431-root\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191516 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191290    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-accelerators-collector-config\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191516 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191304    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a907b903-a15e-487c-87ed-e2852618c431-sys\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191516 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191313    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-tls\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191719 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191381    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a907b903-a15e-487c-87ed-e2852618c431-root\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191719 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191457    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-wtmp\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191719 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191584    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-textfile\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191719 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191630    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a907b903-a15e-487c-87ed-e2852618c431-metrics-client-ca\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191719 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191679    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191958 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191852    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-textfile\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.191958 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.191897    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-accelerators-collector-config\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.192059 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.192044    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a907b903-a15e-487c-87ed-e2852618c431-metrics-client-ca\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.193856 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.193826    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.194172 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.194152    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a907b903-a15e-487c-87ed-e2852618c431-node-exporter-tls\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.200610 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.200585    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgwnn\" (UniqueName: \"kubernetes.io/projected/a907b903-a15e-487c-87ed-e2852618c431-kube-api-access-fgwnn\") pod \"node-exporter-fknj7\" (UID: \"a907b903-a15e-487c-87ed-e2852618c431\") " pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.210801 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.210776    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:51.295714 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.295682    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fknj7"
Sep 08 14:17:51.306242 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:51.306215    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda907b903_a15e_487c_87ed_e2852618c431.slice/crio-7ef992a0200d75ad13291f81bee615478efe6a66c9d68573e30807876abfdb7d WatchSource:0}: Error finding container 7ef992a0200d75ad13291f81bee615478efe6a66c9d68573e30807876abfdb7d: Status 404 returned error can't find the container with id 7ef992a0200d75ad13291f81bee615478efe6a66c9d68573e30807876abfdb7d
Sep 08 14:17:51.310678 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.310650    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:51.412271 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.412238    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:51.660039 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:51.660007    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fknj7" event={"ID":"a907b903-a15e-487c-87ed-e2852618c431","Type":"ContainerStarted","Data":"7ef992a0200d75ad13291f81bee615478efe6a66c9d68573e30807876abfdb7d"}
Sep 08 14:17:52.059621 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.059585    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:17:52.062479 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.062463    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.064913 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.064887    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 08 14:17:52.065039 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.065020    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 08 14:17:52.065294 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.065271    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 08 14:17:52.065379 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.065276    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 08 14:17:52.065442 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.065387    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 08 14:17:52.065442 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.065395    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 08 14:17:52.065442 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.065414    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 08 14:17:52.065591 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.065484    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qh5ls\""
Sep 08 14:17:52.065591 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.065494    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 08 14:17:52.065751 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.065735    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 08 14:17:52.076047 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.076028    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:17:52.199588 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199550    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.199588 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199592    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvc2t\" (UniqueName: \"kubernetes.io/projected/58c3f155-9e8a-47b2-9723-4f95610a26ef-kube-api-access-gvc2t\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.200021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199615    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58c3f155-9e8a-47b2-9723-4f95610a26ef-config-out\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.200021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.200021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199730    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.200021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199757    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c3f155-9e8a-47b2-9723-4f95610a26ef-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.200021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199817    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.200021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.200021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199870    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58c3f155-9e8a-47b2-9723-4f95610a26ef-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.200021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199888    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-config-volume\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.200021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199903    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58c3f155-9e8a-47b2-9723-4f95610a26ef-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.200021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199955    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-web-config\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.200021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.199978    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58c3f155-9e8a-47b2-9723-4f95610a26ef-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.300383 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300345    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58c3f155-9e8a-47b2-9723-4f95610a26ef-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.300383 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300390    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.300601 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300409    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvc2t\" (UniqueName: \"kubernetes.io/projected/58c3f155-9e8a-47b2-9723-4f95610a26ef-kube-api-access-gvc2t\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.300601 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300584    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58c3f155-9e8a-47b2-9723-4f95610a26ef-config-out\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.300684 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300631    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.300684 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300672    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.300777 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c3f155-9e8a-47b2-9723-4f95610a26ef-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.300777 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300759    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.300883 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300804    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.300883 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300835    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58c3f155-9e8a-47b2-9723-4f95610a26ef-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.300883 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300872    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-config-volume\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.301034 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300899    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58c3f155-9e8a-47b2-9723-4f95610a26ef-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.301034 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.300941    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-web-config\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.301808 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.301782    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c3f155-9e8a-47b2-9723-4f95610a26ef-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.302254 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.302229    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58c3f155-9e8a-47b2-9723-4f95610a26ef-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.302560 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.302540    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58c3f155-9e8a-47b2-9723-4f95610a26ef-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.303693 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.303648    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.303693 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.303669    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.304289 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.303761    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58c3f155-9e8a-47b2-9723-4f95610a26ef-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.304289 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.303765    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.304289 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.304241    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.304485 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.304465    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.304706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.304684    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-web-config\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.304783 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.304705    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58c3f155-9e8a-47b2-9723-4f95610a26ef-config-out\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.305124 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.305104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58c3f155-9e8a-47b2-9723-4f95610a26ef-config-volume\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.308911 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.308890    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvc2t\" (UniqueName: \"kubernetes.io/projected/58c3f155-9e8a-47b2-9723-4f95610a26ef-kube-api-access-gvc2t\") pod \"alertmanager-main-0\" (UID: \"58c3f155-9e8a-47b2-9723-4f95610a26ef\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.372459 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.372379    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:17:52.450636 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.450605    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:52.530996 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.530962    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:52.655166 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.655086    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:17:52.658226 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:52.658181    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c3f155_9e8a_47b2_9723_4f95610a26ef.slice/crio-c14d72641a50b685e40c0c8d527cdd8b2769e67a06b3f106cca5d46593c65576 WatchSource:0}: Error finding container c14d72641a50b685e40c0c8d527cdd8b2769e67a06b3f106cca5d46593c65576: Status 404 returned error can't find the container with id c14d72641a50b685e40c0c8d527cdd8b2769e67a06b3f106cca5d46593c65576
Sep 08 14:17:52.668832 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:52.668806    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58c3f155-9e8a-47b2-9723-4f95610a26ef","Type":"ContainerStarted","Data":"c14d72641a50b685e40c0c8d527cdd8b2769e67a06b3f106cca5d46593c65576"}
Sep 08 14:17:53.525876 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.525845    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:53.674755 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.674671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fknj7" event={"ID":"a907b903-a15e-487c-87ed-e2852618c431","Type":"ContainerStarted","Data":"1a9772e3d191dbde8be4751d83638d1f382280647a47f3a03d014fbf88949a1a"}
Sep 08 14:17:53.826025 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.826001    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:53.926562 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.926471    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:53.984148 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.984119    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"]
Sep 08 14:17:53.986697 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.986673    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:53.989148 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.989129    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 08 14:17:53.989323 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.989305    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 08 14:17:53.989409 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.989393    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-37efklrtggk62\""
Sep 08 14:17:53.989469 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.989440    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 08 14:17:53.989805 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.989784    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 08 14:17:53.989871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.989847    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-97mvv\""
Sep 08 14:17:53.989916 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:53.989888    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 08 14:17:54.000073 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.000044    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"]
Sep 08 14:17:54.026245 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.026197    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:54.084434 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.084398    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:54.113817 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.113779    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr5fr\" (UniqueName: \"kubernetes.io/projected/cd2dbb97-1f84-4321-bb36-54d8f9e79767-kube-api-access-qr5fr\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.113817 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.113822    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.114029 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.113844    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-grpc-tls\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.114029 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.113925    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.114029 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.113999    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-tls\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.114140 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.114036    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd2dbb97-1f84-4321-bb36-54d8f9e79767-metrics-client-ca\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.114140 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.114061    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.114140 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.114118    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.214997 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.214910    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.214997 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.214959    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr5fr\" (UniqueName: \"kubernetes.io/projected/cd2dbb97-1f84-4321-bb36-54d8f9e79767-kube-api-access-qr5fr\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.214997 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.214988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.215322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.215013    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-grpc-tls\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.215322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.215048    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.215322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.215108    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-tls\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.215322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.215147    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd2dbb97-1f84-4321-bb36-54d8f9e79767-metrics-client-ca\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.215322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.215177    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.216053 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.215975    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd2dbb97-1f84-4321-bb36-54d8f9e79767-metrics-client-ca\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.217728 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.217697    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.217867 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.217851    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.218171 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.218151    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.218229 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.218165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-grpc-tls\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.218271 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.218250    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.218330 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.218314    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cd2dbb97-1f84-4321-bb36-54d8f9e79767-secret-thanos-querier-tls\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.223193 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.223165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr5fr\" (UniqueName: \"kubernetes.io/projected/cd2dbb97-1f84-4321-bb36-54d8f9e79767-kube-api-access-qr5fr\") pod \"thanos-querier-65964bbcd4-2xhm5\" (UID: \"cd2dbb97-1f84-4321-bb36-54d8f9e79767\") " pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.295815 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.295782    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:54.567749 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.567692    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"]
Sep 08 14:17:54.569886 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:54.569856    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2dbb97_1f84_4321_bb36_54d8f9e79767.slice/crio-08004b356dc08364763e3d32d19eac5edd6aa2c48686b936bd517af15bcbb482 WatchSource:0}: Error finding container 08004b356dc08364763e3d32d19eac5edd6aa2c48686b936bd517af15bcbb482: Status 404 returned error can't find the container with id 08004b356dc08364763e3d32d19eac5edd6aa2c48686b936bd517af15bcbb482
Sep 08 14:17:54.588125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.588096    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:17:54.679728 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.679693    2558 generic.go:356] "Generic (PLEG): container finished" podID="58c3f155-9e8a-47b2-9723-4f95610a26ef" containerID="754b98139ac9b43279047be510d6297f8c97498e2b83a45683681fa229dec6ea" exitCode=0
Sep 08 14:17:54.679902 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.679771    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58c3f155-9e8a-47b2-9723-4f95610a26ef","Type":"ContainerDied","Data":"754b98139ac9b43279047be510d6297f8c97498e2b83a45683681fa229dec6ea"}
Sep 08 14:17:54.681510 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.681405    2558 generic.go:356] "Generic (PLEG): container finished" podID="a907b903-a15e-487c-87ed-e2852618c431" containerID="1a9772e3d191dbde8be4751d83638d1f382280647a47f3a03d014fbf88949a1a" exitCode=0
Sep 08 14:17:54.681510 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.681472    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fknj7" event={"ID":"a907b903-a15e-487c-87ed-e2852618c431","Type":"ContainerDied","Data":"1a9772e3d191dbde8be4751d83638d1f382280647a47f3a03d014fbf88949a1a"}
Sep 08 14:17:54.682716 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.682694    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5" event={"ID":"cd2dbb97-1f84-4321-bb36-54d8f9e79767","Type":"ContainerStarted","Data":"08004b356dc08364763e3d32d19eac5edd6aa2c48686b936bd517af15bcbb482"}
Sep 08 14:17:54.862499 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.862422    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:54.925489 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:54.925449    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:55.689881 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:55.689843    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fknj7" event={"ID":"a907b903-a15e-487c-87ed-e2852618c431","Type":"ContainerStarted","Data":"2ce6dae55e90e1ed2a3b10e4098ed3968e0b5944e975c25f4bdb5715d02a1e1d"}
Sep 08 14:17:55.690302 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:55.689908    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fknj7" event={"ID":"a907b903-a15e-487c-87ed-e2852618c431","Type":"ContainerStarted","Data":"bd658e45d40359853da89dae971edd3e8755d76420d7d52cbd07f5a02dc9ef6b"}
Sep 08 14:17:55.712930 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:55.712884    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fknj7" podStartSLOduration=3.574444078 podStartE2EDuration="5.712868925s" totalImagesPullingTime="2.070965657s" totalInitContainerRuntime="67.45919ms" isStatefulPod=true podCreationTimestamp="2026-09-08 14:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:55.712300319 +0000 UTC m=+90.943231516" watchObservedRunningTime="2026-09-08 14:17:55.712868925 +0000 UTC m=+90.943800121"
Sep 08 14:17:55.725924 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:55.725895    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:55.738172 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:55.738148    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-854976b669-4fdlh"]
Sep 08 14:17:55.827812 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:55.827781    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:55.895020 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:55.894987    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:56.963490 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:56.963453    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:57.058094 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.057983    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:57.200509 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.199647    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:17:57.209388 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.209301    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.212683 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.212379    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1fbaojogikkan\""
Sep 08 14:17:57.213425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.213237    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 08 14:17:57.214027 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.214002    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:57.215168 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.215145    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 08 14:17:57.215407 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.215389    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 08 14:17:57.216015 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.215980    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 08 14:17:57.216134 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.216021    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 08 14:17:57.216134 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.215980    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 08 14:17:57.216343 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.216253    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 08 14:17:57.216343 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.216272    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 08 14:17:57.216696 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.216503    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 08 14:17:57.216696 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.216582    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 08 14:17:57.217729 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.217712    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 08 14:17:57.217881 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.217864    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-77w4r\""
Sep 08 14:17:57.217952 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.217772    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 08 14:17:57.218007 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.217737    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 08 14:17:57.218270 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.218251    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 08 14:17:57.220222 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.219795    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 08 14:17:57.223054 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.223035    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:17:57.344141 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344112    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344254 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344161    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344254 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344188    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344338 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344273    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344338 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344309    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-config\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344338 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344329    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344427 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344349    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344427 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344367    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344427 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344399    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-web-config\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344514 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344428    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-config-out\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344514 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344443    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344514 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344459    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344514 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344474    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344514 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344489    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344646 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344530    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344646 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344551    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpp56\" (UniqueName: \"kubernetes.io/projected/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-kube-api-access-lpp56\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344646 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344582    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344646 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344605    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344646 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344621    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.344787 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.344648    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445355 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445326    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445355 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445358    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpp56\" (UniqueName: \"kubernetes.io/projected/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-kube-api-access-lpp56\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445487 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445382    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445487 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445418    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445487 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445442    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445487 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445483    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445648 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445515    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445648 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445562    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445648 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445588    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445648 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445616    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445650    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-config\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445676    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445702    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445725    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445752    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-web-config\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445795    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-config-out\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.445843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445842    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.446238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445861    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.446238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445882    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.446238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.445987    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.446238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.446128    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.446767 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.446469    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.446767 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.446498    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.446927 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.446869    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.447436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.447049    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.448329 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.447993    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.448716 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.448689    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.449721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.449324    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.449721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.449461    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.449721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.449671    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.449721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.449677    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.450066 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.450032    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.450391 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.450370    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-web-config\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.450540 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.450513    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.450834 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.450813    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.451012 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.450995    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.451125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.451101    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-config-out\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.451654 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.451628    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-config\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.456787 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.456767    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpp56\" (UniqueName: \"kubernetes.io/projected/d18b6f3d-28cf-44dc-8e35-d4405e34ba30-kube-api-access-lpp56\") pod \"prometheus-k8s-0\" (UID: \"d18b6f3d-28cf-44dc-8e35-d4405e34ba30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.534871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.534846    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:17:57.701216 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.701167    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5" event={"ID":"cd2dbb97-1f84-4321-bb36-54d8f9e79767","Type":"ContainerStarted","Data":"f281b5a87eb5386965502fd5448371e22c122e1608c3700e0aac2a167568f115"}
Sep 08 14:17:57.701359 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.701263    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5" event={"ID":"cd2dbb97-1f84-4321-bb36-54d8f9e79767","Type":"ContainerStarted","Data":"5fec2ff3f0b11aaf405cfcdbdc4ecadc6747d382d43c08432b7b0aef2b2516fb"}
Sep 08 14:17:57.701359 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.701299    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5" event={"ID":"cd2dbb97-1f84-4321-bb36-54d8f9e79767","Type":"ContainerStarted","Data":"ce1d8bd3f88689a764af63db4b96f073035f90f233f1b8015a68da82b504c81b"}
Sep 08 14:17:57.704170 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.704148    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58c3f155-9e8a-47b2-9723-4f95610a26ef","Type":"ContainerStarted","Data":"59c5023a35bf2228f82787fd3249ef5deed6178f64a9c6780924391f38123133"}
Sep 08 14:17:57.704279 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.704199    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58c3f155-9e8a-47b2-9723-4f95610a26ef","Type":"ContainerStarted","Data":"7cec47d99a5d969c36def1d4540c84808a04e67f2196667087685ef886f4c0e3"}
Sep 08 14:17:57.704279 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.704247    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58c3f155-9e8a-47b2-9723-4f95610a26ef","Type":"ContainerStarted","Data":"a85555d69dd2a11ce63928d658748f743a213eda9d1ae38c4f9038c277ed0d8c"}
Sep 08 14:17:57.704279 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.704272    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58c3f155-9e8a-47b2-9723-4f95610a26ef","Type":"ContainerStarted","Data":"e24dcd1dd34bb8c353c4276ebcff2621e433ff1879b12d007f8dd3a54ae49a49"}
Sep 08 14:17:57.704369 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.704295    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58c3f155-9e8a-47b2-9723-4f95610a26ef","Type":"ContainerStarted","Data":"40c7d879988ae3f033fc997f05b4b2d396cec70727e036c758f44fa5765d1a70"}
Sep 08 14:17:57.813875 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:57.813680    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:17:57.816505 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:17:57.816471    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18b6f3d_28cf_44dc_8e35_d4405e34ba30.slice/crio-6621205fd22fa88d97c4cd43148b85b4ec1218a5708837376b8a6dfff1a19727 WatchSource:0}: Error finding container 6621205fd22fa88d97c4cd43148b85b4ec1218a5708837376b8a6dfff1a19727: Status 404 returned error can't find the container with id 6621205fd22fa88d97c4cd43148b85b4ec1218a5708837376b8a6dfff1a19727
Sep 08 14:17:58.710928 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:58.710886    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58c3f155-9e8a-47b2-9723-4f95610a26ef","Type":"ContainerStarted","Data":"45f4eb52517a41d646f531e57b04cdcf57d031c7da50c851e5045785a6d4eeb6"}
Sep 08 14:17:58.713792 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:58.713769    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5" event={"ID":"cd2dbb97-1f84-4321-bb36-54d8f9e79767","Type":"ContainerStarted","Data":"91fdd2d16ca5686548482cafe4dfbf61f490e509867052d0d4d8231db8e64c32"}
Sep 08 14:17:58.713891 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:58.713830    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5" event={"ID":"cd2dbb97-1f84-4321-bb36-54d8f9e79767","Type":"ContainerStarted","Data":"70804d3f62f8548d5142f2de582c96aa7b4709d86a3d8507d4ad5c5b23faf7f9"}
Sep 08 14:17:58.713891 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:58.713874    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5" event={"ID":"cd2dbb97-1f84-4321-bb36-54d8f9e79767","Type":"ContainerStarted","Data":"ef3f03c7fb971b41034cb9d033f4348bee8542c773fb96ec0608d163c492a890"}
Sep 08 14:17:58.713988 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:58.713922    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:17:58.715347 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:58.715323    2558 generic.go:356] "Generic (PLEG): container finished" podID="d18b6f3d-28cf-44dc-8e35-d4405e34ba30" containerID="917579682dc846a1cc1ba4426b93b9ce6502e8dc571b117e1b767ab8a8cdb29d" exitCode=0
Sep 08 14:17:58.715423 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:58.715367    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d18b6f3d-28cf-44dc-8e35-d4405e34ba30","Type":"ContainerDied","Data":"917579682dc846a1cc1ba4426b93b9ce6502e8dc571b117e1b767ab8a8cdb29d"}
Sep 08 14:17:58.715457 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:58.715423    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d18b6f3d-28cf-44dc-8e35-d4405e34ba30","Type":"ContainerStarted","Data":"6621205fd22fa88d97c4cd43148b85b4ec1218a5708837376b8a6dfff1a19727"}
Sep 08 14:17:58.740599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:58.740554    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.501282378 podStartE2EDuration="6.740541097s" totalImagesPullingTime="4.220106398s" totalInitContainerRuntime="19.152321ms" isStatefulPod=false podCreationTimestamp="2026-09-08 14:17:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:58.738843716 +0000 UTC m=+93.969774913" watchObservedRunningTime="2026-09-08 14:17:58.740541097 +0000 UTC m=+93.971472294"
Sep 08 14:17:58.798651 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:58.798605    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5" podStartSLOduration=2.758816704 podStartE2EDuration="5.798587345s" totalImagesPullingTime="3.039770641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:17:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:17:58.797146129 +0000 UTC m=+94.028077327" watchObservedRunningTime="2026-09-08 14:17:58.798587345 +0000 UTC m=+94.029518541"
Sep 08 14:17:59.564221 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:59.564177    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:59.668317 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:59.668270    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:17:59.904042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:17:59.903955    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bf8588899-pkqth"]
Sep 08 14:18:00.335861 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:00.335822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:00.436937 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:00.436903    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:00.696607 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:00.696527    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:00.799415 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:00.799375    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:01.688950 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:01.688912    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:01.767015 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:01.766965    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:02.572511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:02.572475    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-92v85"
Sep 08 14:18:02.737044 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:02.737008    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d18b6f3d-28cf-44dc-8e35-d4405e34ba30","Type":"ContainerStarted","Data":"7312b5223fe2015923de11dc4f564ac595008919892eec4272b83a6828611ca9"}
Sep 08 14:18:02.737425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:02.737100    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d18b6f3d-28cf-44dc-8e35-d4405e34ba30","Type":"ContainerStarted","Data":"3f6a597d942218c5279d1b0a2f67f4e610586f3a80309a0df2708f2867c5a2ea"}
Sep 08 14:18:02.737425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:02.737126    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d18b6f3d-28cf-44dc-8e35-d4405e34ba30","Type":"ContainerStarted","Data":"0c009ee413204e4ffc70a5848cf3c1a5d125b9d0fb220b84a3e201d4d2cc8d70"}
Sep 08 14:18:02.737425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:02.737150    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d18b6f3d-28cf-44dc-8e35-d4405e34ba30","Type":"ContainerStarted","Data":"fb1b303e8efb91547032b23394d99c638dc91f4625098f6654699df23130218a"}
Sep 08 14:18:02.737425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:02.737179    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d18b6f3d-28cf-44dc-8e35-d4405e34ba30","Type":"ContainerStarted","Data":"194f5806405d7aab9b66aeeb839a54d899321a05db1dd3daf1dfd6183c1d34f6"}
Sep 08 14:18:02.737425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:02.737218    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d18b6f3d-28cf-44dc-8e35-d4405e34ba30","Type":"ContainerStarted","Data":"76ff8329c366e9843da49611cdd5dde6b2f46c9321a14ad976bc3fddba33282e"}
Sep 08 14:18:02.765773 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:02.765174    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.621469625 podStartE2EDuration="5.765154261s" totalImagesPullingTime="3.124501843s" totalInitContainerRuntime="19.182793ms" isStatefulPod=false podCreationTimestamp="2026-09-08 14:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:18:02.764331072 +0000 UTC m=+97.995262294" watchObservedRunningTime="2026-09-08 14:18:02.765154261 +0000 UTC m=+97.996085463"
Sep 08 14:18:04.726394 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:04.726365    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-65964bbcd4-2xhm5"
Sep 08 14:18:07.535975 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:07.535935    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:18:10.128430 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:10.128397    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:10.194125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:10.194093    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:14.022938 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:14.022899    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:14.087701 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:14.087667    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:15.739500 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:15.739462    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:15.795704 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:15.795665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:20.759628 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:20.759582    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-854976b669-4fdlh" podUID="8f194f86-4302-46b1-b765-094007b9b1b4" containerName="registry" containerID="cri-o://d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048" gracePeriod=30
Sep 08 14:18:20.999216 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:20.999181    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:18:21.056887 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.056816    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-registry-certificates\") pod \"8f194f86-4302-46b1-b765-094007b9b1b4\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") "
Sep 08 14:18:21.056887 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.056853    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fc8wg\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-kube-api-access-fc8wg\") pod \"8f194f86-4302-46b1-b765-094007b9b1b4\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") "
Sep 08 14:18:21.056887 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.056875    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-bound-sa-token\") pod \"8f194f86-4302-46b1-b765-094007b9b1b4\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") "
Sep 08 14:18:21.057098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.056899    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-image-registry-private-configuration\") pod \"8f194f86-4302-46b1-b765-094007b9b1b4\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") "
Sep 08 14:18:21.057098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.056916    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-installation-pull-secrets\") pod \"8f194f86-4302-46b1-b765-094007b9b1b4\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") "
Sep 08 14:18:21.057442 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.057360    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-trusted-ca\") pod \"8f194f86-4302-46b1-b765-094007b9b1b4\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") "
Sep 08 14:18:21.057442 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.057429    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls\") pod \"8f194f86-4302-46b1-b765-094007b9b1b4\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") "
Sep 08 14:18:21.057616 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.057550    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f194f86-4302-46b1-b765-094007b9b1b4-ca-trust-extracted\") pod \"8f194f86-4302-46b1-b765-094007b9b1b4\" (UID: \"8f194f86-4302-46b1-b765-094007b9b1b4\") "
Sep 08 14:18:21.057676 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.057649    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8f194f86-4302-46b1-b765-094007b9b1b4" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:18:21.057774 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.057744    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8f194f86-4302-46b1-b765-094007b9b1b4" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:18:21.059124 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.058616    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-trusted-ca\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:21.059124 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.058651    2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f194f86-4302-46b1-b765-094007b9b1b4-registry-certificates\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:21.061111 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.060493    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8f194f86-4302-46b1-b765-094007b9b1b4" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:18:21.061238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.061169    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-kube-api-access-fc8wg" (OuterVolumeSpecName: "kube-api-access-fc8wg") pod "8f194f86-4302-46b1-b765-094007b9b1b4" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4"). InnerVolumeSpecName "kube-api-access-fc8wg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:18:21.065590 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.065550    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8f194f86-4302-46b1-b765-094007b9b1b4" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:18:21.065684 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.065597    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8f194f86-4302-46b1-b765-094007b9b1b4" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:18:21.065684 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.065656    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8f194f86-4302-46b1-b765-094007b9b1b4" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:18:21.068620 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.068598    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f194f86-4302-46b1-b765-094007b9b1b4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8f194f86-4302-46b1-b765-094007b9b1b4" (UID: "8f194f86-4302-46b1-b765-094007b9b1b4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:18:21.159296 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.159251    2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-registry-tls\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:21.159296 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.159293    2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f194f86-4302-46b1-b765-094007b9b1b4-ca-trust-extracted\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:21.159296 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.159304    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fc8wg\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-kube-api-access-fc8wg\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:21.159516 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.159314    2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f194f86-4302-46b1-b765-094007b9b1b4-bound-sa-token\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:21.159516 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.159324    2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-image-registry-private-configuration\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:21.159516 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.159333    2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f194f86-4302-46b1-b765-094007b9b1b4-installation-pull-secrets\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:21.812327 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.812290    2558 generic.go:356] "Generic (PLEG): container finished" podID="8f194f86-4302-46b1-b765-094007b9b1b4" containerID="d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048" exitCode=0
Sep 08 14:18:21.812702 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.812345    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-854976b669-4fdlh"
Sep 08 14:18:21.812702 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.812372    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-854976b669-4fdlh" event={"ID":"8f194f86-4302-46b1-b765-094007b9b1b4","Type":"ContainerDied","Data":"d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048"}
Sep 08 14:18:21.812702 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.812423    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-854976b669-4fdlh" event={"ID":"8f194f86-4302-46b1-b765-094007b9b1b4","Type":"ContainerDied","Data":"7de6a3bfbacb8ff25c6d2750b2c634ae5a6cfb5344a9677929544eb66045b4d6"}
Sep 08 14:18:21.812702 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.812446    2558 scope.go:117] "RemoveContainer" containerID="d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048"
Sep 08 14:18:21.820569 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.820547    2558 scope.go:117] "RemoveContainer" containerID="d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048"
Sep 08 14:18:21.820849 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:18:21.820830    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048\": container with ID starting with d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048 not found: ID does not exist" containerID="d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048"
Sep 08 14:18:21.820917 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.820859    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048"} err="failed to get container status \"d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048\": rpc error: code = NotFound desc = could not find container \"d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048\": container with ID starting with d327ad90eec95d80bcea42fdda197b0d9a962dc4d9d50799bd172a0dc5bbc048 not found: ID does not exist"
Sep 08 14:18:21.831743 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.831719    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-854976b669-4fdlh"]
Sep 08 14:18:21.835632 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:21.835609    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-854976b669-4fdlh"]
Sep 08 14:18:23.285863 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:23.285828    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f194f86-4302-46b1-b765-094007b9b1b4" path="/var/lib/kubelet/pods/8f194f86-4302-46b1-b765-094007b9b1b4/volumes"
Sep 08 14:18:24.932768 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:24.932708    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bf8588899-pkqth" podUID="a7f1e1d8-0294-4874-9ed3-289d58f5714b" containerName="console" containerID="cri-o://a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f" gracePeriod=15
Sep 08 14:18:25.173501 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.173477    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:18:25.295157 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.295130    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-service-ca\") pod \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") "
Sep 08 14:18:25.295157 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.295157    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-config\") pod \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") "
Sep 08 14:18:25.295393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.295200    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-serving-cert\") pod \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") "
Sep 08 14:18:25.295393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.295251    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j27sv\" (UniqueName: \"kubernetes.io/projected/a7f1e1d8-0294-4874-9ed3-289d58f5714b-kube-api-access-j27sv\") pod \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") "
Sep 08 14:18:25.295393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.295301    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-oauth-config\") pod \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") "
Sep 08 14:18:25.295393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.295373    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-oauth-serving-cert\") pod \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\" (UID: \"a7f1e1d8-0294-4874-9ed3-289d58f5714b\") "
Sep 08 14:18:25.295601 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.295573    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-service-ca" (OuterVolumeSpecName: "service-ca") pod "a7f1e1d8-0294-4874-9ed3-289d58f5714b" (UID: "a7f1e1d8-0294-4874-9ed3-289d58f5714b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:18:25.295734 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.295607    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-config" (OuterVolumeSpecName: "console-config") pod "a7f1e1d8-0294-4874-9ed3-289d58f5714b" (UID: "a7f1e1d8-0294-4874-9ed3-289d58f5714b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:18:25.295881 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.295859    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a7f1e1d8-0294-4874-9ed3-289d58f5714b" (UID: "a7f1e1d8-0294-4874-9ed3-289d58f5714b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:18:25.297691 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.297662    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f1e1d8-0294-4874-9ed3-289d58f5714b-kube-api-access-j27sv" (OuterVolumeSpecName: "kube-api-access-j27sv") pod "a7f1e1d8-0294-4874-9ed3-289d58f5714b" (UID: "a7f1e1d8-0294-4874-9ed3-289d58f5714b"). InnerVolumeSpecName "kube-api-access-j27sv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:18:25.297783 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.297707    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a7f1e1d8-0294-4874-9ed3-289d58f5714b" (UID: "a7f1e1d8-0294-4874-9ed3-289d58f5714b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:18:25.297783 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.297731    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a7f1e1d8-0294-4874-9ed3-289d58f5714b" (UID: "a7f1e1d8-0294-4874-9ed3-289d58f5714b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:18:25.395914 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.395885    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-oauth-serving-cert\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:25.395914 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.395910    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-service-ca\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:25.395914 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.395919    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-config\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:25.396118 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.395927    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-serving-cert\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:25.396118 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.395935    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j27sv\" (UniqueName: \"kubernetes.io/projected/a7f1e1d8-0294-4874-9ed3-289d58f5714b-kube-api-access-j27sv\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:25.396118 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.395943    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7f1e1d8-0294-4874-9ed3-289d58f5714b-console-oauth-config\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:18:25.830671 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.830630    2558 generic.go:356] "Generic (PLEG): container finished" podID="3fc3293c-9e19-4a82-b33b-dea02080290f" containerID="63771aec6afe05efc1b8f330eb4458f48cbc1f1cf9d3431576dc45792f5c8a22" exitCode=0
Sep 08 14:18:25.830850 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.830704    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb" event={"ID":"3fc3293c-9e19-4a82-b33b-dea02080290f","Type":"ContainerDied","Data":"63771aec6afe05efc1b8f330eb4458f48cbc1f1cf9d3431576dc45792f5c8a22"}
Sep 08 14:18:25.831084 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.831059    2558 scope.go:117] "RemoveContainer" containerID="63771aec6afe05efc1b8f330eb4458f48cbc1f1cf9d3431576dc45792f5c8a22"
Sep 08 14:18:25.832034 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.832011    2558 generic.go:356] "Generic (PLEG): container finished" podID="a7f1e1d8-0294-4874-9ed3-289d58f5714b" containerID="a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f" exitCode=0
Sep 08 14:18:25.832131 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.832066    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf8588899-pkqth"
Sep 08 14:18:25.832131 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.832090    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf8588899-pkqth" event={"ID":"a7f1e1d8-0294-4874-9ed3-289d58f5714b","Type":"ContainerDied","Data":"a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f"}
Sep 08 14:18:25.832253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.832140    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf8588899-pkqth" event={"ID":"a7f1e1d8-0294-4874-9ed3-289d58f5714b","Type":"ContainerDied","Data":"d768f83817268b2f03f4b829a42f1cadb446db0a9d9cea731398212721eab7df"}
Sep 08 14:18:25.832253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.832164    2558 scope.go:117] "RemoveContainer" containerID="a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f"
Sep 08 14:18:25.843412 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.843390    2558 scope.go:117] "RemoveContainer" containerID="a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f"
Sep 08 14:18:25.843670 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:18:25.843653    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f\": container with ID starting with a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f not found: ID does not exist" containerID="a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f"
Sep 08 14:18:25.843717 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.843677    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f"} err="failed to get container status \"a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f\": rpc error: code = NotFound desc = could not find container \"a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f\": container with ID starting with a1020a53de33f963ad977ed44146248557c3dbb2a75c1fc44cae6e862a7e875f not found: ID does not exist"
Sep 08 14:18:25.867253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.867226    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bf8588899-pkqth"]
Sep 08 14:18:25.873523 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:25.873501    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bf8588899-pkqth"]
Sep 08 14:18:26.837609 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:26.837573    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rn2gb" event={"ID":"3fc3293c-9e19-4a82-b33b-dea02080290f","Type":"ContainerStarted","Data":"4f51449fd0d2d02f8835bb8465d107cf4dd59beb7f6ff2dde09e8981b7f36852"}
Sep 08 14:18:27.285806 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:27.285775    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7f1e1d8-0294-4874-9ed3-289d58f5714b" path="/var/lib/kubelet/pods/a7f1e1d8-0294-4874-9ed3-289d58f5714b/volumes"
Sep 08 14:18:28.714870 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:28.714827    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:30.732953 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:30.732917    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:30.796090 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:30.796040    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:35.613378 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:35.613347    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-585d697d6-dm2q6_595b339a-11fd-45dc-a950-63dcd5236b02/router/0.log"
Sep 08 14:18:35.620196 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:35.620172    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gxn5d_a1d1aacf-d0b7-4da2-bd12-beaefcc94966/serve-healthcheck-canary/0.log"
Sep 08 14:18:39.354757 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:39.354728    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:39.455920 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:39.455884    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:39.557579 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:39.557547    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:39.888003 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:39.887967    2558 generic.go:356] "Generic (PLEG): container finished" podID="d3039d4d-4454-4649-8e1a-6dc0f5bfe51b" containerID="bd27e5ae75fc4a599fb71d5cdad03fc4cd67c6f04d10c478acbe2abb6c112164" exitCode=0
Sep 08 14:18:39.888165 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:39.888040    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r" event={"ID":"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b","Type":"ContainerDied","Data":"bd27e5ae75fc4a599fb71d5cdad03fc4cd67c6f04d10c478acbe2abb6c112164"}
Sep 08 14:18:39.888381 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:39.888368    2558 scope.go:117] "RemoveContainer" containerID="bd27e5ae75fc4a599fb71d5cdad03fc4cd67c6f04d10c478acbe2abb6c112164"
Sep 08 14:18:40.893754 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:40.893717    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cxl6r" event={"ID":"d3039d4d-4454-4649-8e1a-6dc0f5bfe51b","Type":"ContainerStarted","Data":"38c541139290d368a0f20d18506c3cc36b74a9053e39b83fc032d6d22ec59e17"}
Sep 08 14:18:48.215026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:48.214988    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:18:57.535479 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:57.535439    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:18:57.551142 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:57.551112    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:18:57.979706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:18:57.979617    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:19:07.102923 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:19:07.102886    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:19:07.163893 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:19:07.163858    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:19:19.137072 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:19:19.136961    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:19:19.232130 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:19:19.232092    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:19:30.410998 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:19:30.410932    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:19:30.512734 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:19:30.512703    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:19:30.868707 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:19:30.868671    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:19:30.937574 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:19:30.937547    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:19:56.747924 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:19:56.747889    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:19:56.804900 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:19:56.804869    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:20:05.912783 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:20:05.912746    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:20:05.969118 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:20:05.969091    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:20:08.172698 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:20:08.172665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:20:49.690729 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:20:49.690646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:08.943565 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:08.943518    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:09.040479 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:09.040448    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:25.204670 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:25.204639    2558 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 14:21:30.719654 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:30.719621    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:31.914924 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:31.914888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:34.319061 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:34.319028    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:34.406433 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:34.406358    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:36.379123 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:36.379062    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:36.485833 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:36.485795    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:36.683724 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:36.683641    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:38.311437 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:38.311399    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:38.825758 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:38.825728    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:39.195967 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:39.195895    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:41.298793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:41.298755    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:41.401387 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:41.401349    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:46.385424 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:46.385394    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:46.679027 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:46.678935    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:46.737184 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:46.737151    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:47.901762 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:47.901729    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:48.002963 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:48.002931    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:55.748739 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:55.748706    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:55.809916 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:55.809888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:21:57.799988 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:21:57.799950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:11.824821 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:11.824787    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:11.924439 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:11.924406    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:12.437379 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.437346    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:12.681378 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.681343    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:12.792310 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.792283    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:12.847005 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.846972    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"]
Sep 08 14:22:12.847399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.847295    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f194f86-4302-46b1-b765-094007b9b1b4" containerName="registry"
Sep 08 14:22:12.847399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.847309    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f194f86-4302-46b1-b765-094007b9b1b4" containerName="registry"
Sep 08 14:22:12.847399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.847325    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7f1e1d8-0294-4874-9ed3-289d58f5714b" containerName="console"
Sep 08 14:22:12.847399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.847330    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7f1e1d8-0294-4874-9ed3-289d58f5714b" containerName="console"
Sep 08 14:22:12.847399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.847384    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7f1e1d8-0294-4874-9ed3-289d58f5714b" containerName="console"
Sep 08 14:22:12.847399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.847393    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f194f86-4302-46b1-b765-094007b9b1b4" containerName="registry"
Sep 08 14:22:12.849366 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.849351    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:12.852693 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.852667    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 14:22:12.852854 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.852669    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 14:22:12.852975 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.852679    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 14:22:12.852975 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.852680    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 14:22:12.853095 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.852758    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-js9cr\""
Sep 08 14:22:12.864228 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.864194    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"]
Sep 08 14:22:12.928750 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.928719    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-webhook-cert\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:12.928945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.928764    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-tmp\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:12.928945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.928826    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:12.928945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.928859    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljdbt\" (UniqueName: \"kubernetes.io/projected/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-kube-api-access-ljdbt\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:12.928945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.928926    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-manifests\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:12.953250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:12.953199    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:13.029438 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.029407    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:13.029438 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.029445    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljdbt\" (UniqueName: \"kubernetes.io/projected/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-kube-api-access-ljdbt\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:13.029679 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.029479    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-manifests\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:13.029679 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.029516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-webhook-cert\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:13.029679 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.029563    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-tmp\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:13.029920 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.029897    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-manifests\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:13.031893 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.031869    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-tmp\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:13.031998 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.031914    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:13.032059 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.032022    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-webhook-cert\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:13.038947 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.038924    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljdbt\" (UniqueName: \"kubernetes.io/projected/6254bbfb-4506-4cad-ba3e-aa970ec3bd4e-kube-api-access-ljdbt\") pod \"opendatahub-operator-controller-manager-566d57d45b-wzqgs\" (UID: \"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e\") " pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:13.057119 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.057063    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:13.159052 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.159018    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:13.160000 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.159975    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:13.316296 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.316195    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:13.413916 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.413884    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:13.440849 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.440825    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"]
Sep 08 14:22:13.443039 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:22:13.443012    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6254bbfb_4506_4cad_ba3e_aa970ec3bd4e.slice/crio-9d4143a5b652d2ac7f40ecc66cbfc43c2249c5d18ac0e15a41da856f0eb8dd2b WatchSource:0}: Error finding container 9d4143a5b652d2ac7f40ecc66cbfc43c2249c5d18ac0e15a41da856f0eb8dd2b: Status 404 returned error can't find the container with id 9d4143a5b652d2ac7f40ecc66cbfc43c2249c5d18ac0e15a41da856f0eb8dd2b
Sep 08 14:22:13.445313 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.445297    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:22:13.711960 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.711879    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs" event={"ID":"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e","Type":"ContainerStarted","Data":"9d4143a5b652d2ac7f40ecc66cbfc43c2249c5d18ac0e15a41da856f0eb8dd2b"}
Sep 08 14:22:13.941811 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:13.941777    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:14.049423 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:14.049356    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:16.727681 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:16.727593    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs" event={"ID":"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e","Type":"ContainerStarted","Data":"ff42870e21087b456d9f66f0f20ce9b5f9c269351de46c20f277a40dad9f3f87"}
Sep 08 14:22:17.732862 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:17.732828    2558 generic.go:356] "Generic (PLEG): container finished" podID="6254bbfb-4506-4cad-ba3e-aa970ec3bd4e" containerID="ff42870e21087b456d9f66f0f20ce9b5f9c269351de46c20f277a40dad9f3f87" exitCode=0
Sep 08 14:22:17.733330 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:17.732900    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs" event={"ID":"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e","Type":"ContainerDied","Data":"ff42870e21087b456d9f66f0f20ce9b5f9c269351de46c20f277a40dad9f3f87"}
Sep 08 14:22:18.738264 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:18.738226    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs" event={"ID":"6254bbfb-4506-4cad-ba3e-aa970ec3bd4e","Type":"ContainerStarted","Data":"f7e39d9f075a1c340599b62c6b391507ddfbd76f2f87d60c98da43d0e34ad06d"}
Sep 08 14:22:18.738634 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:18.738335    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:18.760360 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:18.760300    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs" podStartSLOduration=2.424189036 podStartE2EDuration="6.760279628s" totalImagesPullingTime="3.244862101s" totalInitContainerRuntime="1.091228491s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:22:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:22:18.757920208 +0000 UTC m=+353.988851426" watchObservedRunningTime="2026-09-08 14:22:18.760279628 +0000 UTC m=+353.991210827"
Sep 08 14:22:29.743854 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:29.743821    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-566d57d45b-wzqgs"
Sep 08 14:22:33.170786 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.170748    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:33.331737 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.331692    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:33.557508 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.557466    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:33.595546 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.595511    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"]
Sep 08 14:22:33.605147 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.605124    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:33.607421 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.607393    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 08 14:22:33.607581 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.607397    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-v5rm7\""
Sep 08 14:22:33.608614 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.608593    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 08 14:22:33.610441 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.610417    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"]
Sep 08 14:22:33.711482 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.711445    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59758a79-a64a-4717-905c-88d7f6751099-tmp\") pod \"kube-auth-proxy-5df94dbdbc-kpft5\" (UID: \"59758a79-a64a-4717-905c-88d7f6751099\") " pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:33.711646 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.711505    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59758a79-a64a-4717-905c-88d7f6751099-tls-certs\") pod \"kube-auth-proxy-5df94dbdbc-kpft5\" (UID: \"59758a79-a64a-4717-905c-88d7f6751099\") " pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:33.711646 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.711568    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd88k\" (UniqueName: \"kubernetes.io/projected/59758a79-a64a-4717-905c-88d7f6751099-kube-api-access-nd88k\") pod \"kube-auth-proxy-5df94dbdbc-kpft5\" (UID: \"59758a79-a64a-4717-905c-88d7f6751099\") " pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:33.812082 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.811986    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59758a79-a64a-4717-905c-88d7f6751099-tls-certs\") pod \"kube-auth-proxy-5df94dbdbc-kpft5\" (UID: \"59758a79-a64a-4717-905c-88d7f6751099\") " pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:33.812082 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.812041    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd88k\" (UniqueName: \"kubernetes.io/projected/59758a79-a64a-4717-905c-88d7f6751099-kube-api-access-nd88k\") pod \"kube-auth-proxy-5df94dbdbc-kpft5\" (UID: \"59758a79-a64a-4717-905c-88d7f6751099\") " pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:33.812381 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.812106    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59758a79-a64a-4717-905c-88d7f6751099-tmp\") pod \"kube-auth-proxy-5df94dbdbc-kpft5\" (UID: \"59758a79-a64a-4717-905c-88d7f6751099\") " pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:33.812381 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:22:33.812158    2558 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 08 14:22:33.812381 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:22:33.812276    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59758a79-a64a-4717-905c-88d7f6751099-tls-certs podName:59758a79-a64a-4717-905c-88d7f6751099 nodeName:}" failed. No retries permitted until 2026-09-08 14:22:34.312255199 +0000 UTC m=+369.543186373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/59758a79-a64a-4717-905c-88d7f6751099-tls-certs") pod "kube-auth-proxy-5df94dbdbc-kpft5" (UID: "59758a79-a64a-4717-905c-88d7f6751099") : secret "kube-auth-proxy-tls" not found
Sep 08 14:22:33.814292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.814263    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59758a79-a64a-4717-905c-88d7f6751099-tmp\") pod \"kube-auth-proxy-5df94dbdbc-kpft5\" (UID: \"59758a79-a64a-4717-905c-88d7f6751099\") " pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:33.820604 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:33.820583    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd88k\" (UniqueName: \"kubernetes.io/projected/59758a79-a64a-4717-905c-88d7f6751099-kube-api-access-nd88k\") pod \"kube-auth-proxy-5df94dbdbc-kpft5\" (UID: \"59758a79-a64a-4717-905c-88d7f6751099\") " pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:34.089735 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:34.089649    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:34.316687 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:34.316648    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59758a79-a64a-4717-905c-88d7f6751099-tls-certs\") pod \"kube-auth-proxy-5df94dbdbc-kpft5\" (UID: \"59758a79-a64a-4717-905c-88d7f6751099\") " pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:34.319106 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:34.319085    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59758a79-a64a-4717-905c-88d7f6751099-tls-certs\") pod \"kube-auth-proxy-5df94dbdbc-kpft5\" (UID: \"59758a79-a64a-4717-905c-88d7f6751099\") " pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:34.515249 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:34.515197    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"
Sep 08 14:22:34.676647 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:34.676615    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:34.736253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:34.736224    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:34.782958 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:34.782932    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5"]
Sep 08 14:22:34.785708 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:22:34.785681    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59758a79_a64a_4717_905c_88d7f6751099.slice/crio-876fc76bc0d7164f20e6957a0fab71f1255bc169baf16a38162d6af649453d00 WatchSource:0}: Error finding container 876fc76bc0d7164f20e6957a0fab71f1255bc169baf16a38162d6af649453d00: Status 404 returned error can't find the container with id 876fc76bc0d7164f20e6957a0fab71f1255bc169baf16a38162d6af649453d00
Sep 08 14:22:34.804048 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:34.804002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5" event={"ID":"59758a79-a64a-4717-905c-88d7f6751099","Type":"ContainerStarted","Data":"876fc76bc0d7164f20e6957a0fab71f1255bc169baf16a38162d6af649453d00"}
Sep 08 14:22:35.696769 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:35.696734    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:37.820325 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:37.820290    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5" event={"ID":"59758a79-a64a-4717-905c-88d7f6751099","Type":"ContainerStarted","Data":"453a8529ed780e09b5fb996aac20502e00b9497e77607eaf9b9bd77ab2ab28c8"}
Sep 08 14:22:37.836948 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:37.836893    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5df94dbdbc-kpft5" podStartSLOduration=1.9922376929999999 podStartE2EDuration="4.8368784s" totalImagesPullingTime="2.844640707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:22:37.835585067 +0000 UTC m=+373.066516257" watchObservedRunningTime="2026-09-08 14:22:37.8368784 +0000 UTC m=+373.067809597"
Sep 08 14:22:38.784469 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:38.784433    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:38.882158 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:38.882114    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:39.074839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:39.074734    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:39.172494 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:39.172456    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:39.304534 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:39.304499    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:44.227887 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.227851    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kwhrf"]
Sep 08 14:22:44.232285 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.232268    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf"
Sep 08 14:22:44.234933 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.234903    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 08 14:22:44.235048 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.234912    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-p2s9c\""
Sep 08 14:22:44.239817 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.239793    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kwhrf"]
Sep 08 14:22:44.310134 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.310103    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:44.406749 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.406717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj7tx\" (UniqueName: \"kubernetes.io/projected/e4bb59a3-1c4c-427f-9bea-3a5179e00133-kube-api-access-nj7tx\") pod \"odh-model-controller-7998c67dc4-kwhrf\" (UID: \"e4bb59a3-1c4c-427f-9bea-3a5179e00133\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf"
Sep 08 14:22:44.406911 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.406776    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4bb59a3-1c4c-427f-9bea-3a5179e00133-cert\") pod \"odh-model-controller-7998c67dc4-kwhrf\" (UID: \"e4bb59a3-1c4c-427f-9bea-3a5179e00133\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf"
Sep 08 14:22:44.407580 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.407528    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:44.507438 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.507399    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj7tx\" (UniqueName: \"kubernetes.io/projected/e4bb59a3-1c4c-427f-9bea-3a5179e00133-kube-api-access-nj7tx\") pod \"odh-model-controller-7998c67dc4-kwhrf\" (UID: \"e4bb59a3-1c4c-427f-9bea-3a5179e00133\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf"
Sep 08 14:22:44.507637 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.507513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4bb59a3-1c4c-427f-9bea-3a5179e00133-cert\") pod \"odh-model-controller-7998c67dc4-kwhrf\" (UID: \"e4bb59a3-1c4c-427f-9bea-3a5179e00133\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf"
Sep 08 14:22:44.507715 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:22:44.507666    2558 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 08 14:22:44.507769 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:22:44.507726    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4bb59a3-1c4c-427f-9bea-3a5179e00133-cert podName:e4bb59a3-1c4c-427f-9bea-3a5179e00133 nodeName:}" failed. No retries permitted until 2026-09-08 14:22:45.007711313 +0000 UTC m=+380.238642489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e4bb59a3-1c4c-427f-9bea-3a5179e00133-cert") pod "odh-model-controller-7998c67dc4-kwhrf" (UID: "e4bb59a3-1c4c-427f-9bea-3a5179e00133") : secret "odh-model-controller-webhook-cert" not found
Sep 08 14:22:44.508281 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.508261    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:44.519172 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.519147    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj7tx\" (UniqueName: \"kubernetes.io/projected/e4bb59a3-1c4c-427f-9bea-3a5179e00133-kube-api-access-nj7tx\") pod \"odh-model-controller-7998c67dc4-kwhrf\" (UID: \"e4bb59a3-1c4c-427f-9bea-3a5179e00133\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf"
Sep 08 14:22:44.766135 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.766064    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:44.851194 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:44.851163    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:45.011951 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:45.011914    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4bb59a3-1c4c-427f-9bea-3a5179e00133-cert\") pod \"odh-model-controller-7998c67dc4-kwhrf\" (UID: \"e4bb59a3-1c4c-427f-9bea-3a5179e00133\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf"
Sep 08 14:22:45.014370 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:45.014351    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4bb59a3-1c4c-427f-9bea-3a5179e00133-cert\") pod \"odh-model-controller-7998c67dc4-kwhrf\" (UID: \"e4bb59a3-1c4c-427f-9bea-3a5179e00133\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf"
Sep 08 14:22:45.082638 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:45.082558    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:45.143080 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:45.143046    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf"
Sep 08 14:22:45.422251 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:45.422226    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kwhrf"]
Sep 08 14:22:45.425006 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:22:45.424981    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4bb59a3_1c4c_427f_9bea_3a5179e00133.slice/crio-1922a092ae3002302a404b48a573fa1af57953da556cde2219f16ca9b556d5f8 WatchSource:0}: Error finding container 1922a092ae3002302a404b48a573fa1af57953da556cde2219f16ca9b556d5f8: Status 404 returned error can't find the container with id 1922a092ae3002302a404b48a573fa1af57953da556cde2219f16ca9b556d5f8
Sep 08 14:22:45.856201 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:45.856154    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf" event={"ID":"e4bb59a3-1c4c-427f-9bea-3a5179e00133","Type":"ContainerStarted","Data":"1922a092ae3002302a404b48a573fa1af57953da556cde2219f16ca9b556d5f8"}
Sep 08 14:22:46.854196 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.854161    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76785bcfcc-lnsvb"]
Sep 08 14:22:46.862566 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.862540    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:46.868458 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.868373    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76785bcfcc-lnsvb"]
Sep 08 14:22:46.869520 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.868946    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 08 14:22:46.869520 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.869048    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 08 14:22:46.869520 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.869161    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 08 14:22:46.869520 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.869270    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 08 14:22:46.869520 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.869444    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 08 14:22:46.869835 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.869628    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 08 14:22:46.871440 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.871185    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 08 14:22:46.871440 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.871255    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kd8vx\""
Sep 08 14:22:46.878479 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.878461    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 08 14:22:46.909145 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.909103    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:46.928964 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.928930    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-oauth-serving-cert\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:46.929133 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.929002    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-console-config\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:46.929133 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.929058    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-service-ca\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:46.929133 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.929104    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-console-oauth-config\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:46.929133 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.929128    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtc2t\" (UniqueName: \"kubernetes.io/projected/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-kube-api-access-qtc2t\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:46.929369 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.929190    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-trusted-ca-bundle\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:46.929369 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.929234    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-console-serving-cert\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:46.965520 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:46.965439    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:47.029900 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.029866    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-console-config\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.030088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.029928    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-service-ca\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.030088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.029970    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-console-oauth-config\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.030088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.029988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtc2t\" (UniqueName: \"kubernetes.io/projected/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-kube-api-access-qtc2t\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.030088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.030039    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-trusted-ca-bundle\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.030088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.030057    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-console-serving-cert\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.030391 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.030110    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-oauth-serving-cert\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.030757 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.030691    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-console-config\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.030886 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.030865    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-service-ca\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.030989 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.030967    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-oauth-serving-cert\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.031229 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.031181    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-trusted-ca-bundle\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.032999 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.032956    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-console-oauth-config\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.033291 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.033263    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-console-serving-cert\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.039650 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.039628    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtc2t\" (UniqueName: \"kubernetes.io/projected/fcf7f7f5-5fdf-4eab-a001-9a70b2927928-kube-api-access-qtc2t\") pod \"console-76785bcfcc-lnsvb\" (UID: \"fcf7f7f5-5fdf-4eab-a001-9a70b2927928\") " pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.180096 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.180012    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:47.766444 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:47.766413    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76785bcfcc-lnsvb"]
Sep 08 14:22:48.169584 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:22:48.169514    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf7f7f5_5fdf_4eab_a001_9a70b2927928.slice/crio-f0020549d6b3742f5d98961a60c79dd37572fc87fe7fb87f259e9c7ff9da27ec WatchSource:0}: Error finding container f0020549d6b3742f5d98961a60c79dd37572fc87fe7fb87f259e9c7ff9da27ec: Status 404 returned error can't find the container with id f0020549d6b3742f5d98961a60c79dd37572fc87fe7fb87f259e9c7ff9da27ec
Sep 08 14:22:48.874041 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:48.874002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf" event={"ID":"e4bb59a3-1c4c-427f-9bea-3a5179e00133","Type":"ContainerStarted","Data":"9495be29b9f0629e04aa4f57b8f2c62dee8167e9d195ea6a948f263810da0618"}
Sep 08 14:22:48.874263 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:48.874120    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf"
Sep 08 14:22:48.875591 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:48.875562    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76785bcfcc-lnsvb" event={"ID":"fcf7f7f5-5fdf-4eab-a001-9a70b2927928","Type":"ContainerStarted","Data":"00da287c9db9b5675da49c788f6d28b1e4242ccf11db099d9e921f225488cf14"}
Sep 08 14:22:48.875692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:48.875606    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76785bcfcc-lnsvb" event={"ID":"fcf7f7f5-5fdf-4eab-a001-9a70b2927928","Type":"ContainerStarted","Data":"f0020549d6b3742f5d98961a60c79dd37572fc87fe7fb87f259e9c7ff9da27ec"}
Sep 08 14:22:48.894897 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:48.894846    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf" podStartSLOduration=2.098072549 podStartE2EDuration="4.894828265s" totalImagesPullingTime="2.796755716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:22:48.894629795 +0000 UTC m=+384.125560992" watchObservedRunningTime="2026-09-08 14:22:48.894828265 +0000 UTC m=+384.125759463"
Sep 08 14:22:48.913248 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:48.913186    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76785bcfcc-lnsvb" podStartSLOduration=2.913171017 podStartE2EDuration="2.913171017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:22:48.911309931 +0000 UTC m=+384.142241131" watchObservedRunningTime="2026-09-08 14:22:48.913171017 +0000 UTC m=+384.144102214"
Sep 08 14:22:51.612630 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.612588    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"]
Sep 08 14:22:51.616693 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.616669    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.619344 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.619324    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 08 14:22:51.620791 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.620647    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 08 14:22:51.620791 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.620647    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 08 14:22:51.620791 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.620707    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wkspg\""
Sep 08 14:22:51.620791 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.620764    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 08 14:22:51.621006 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.620949    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 08 14:22:51.625078 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.625056    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"]
Sep 08 14:22:51.674104 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.674071    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnzxw\" (UniqueName: \"kubernetes.io/projected/86970df8-d442-47d7-ba84-d7a7c84054bc-kube-api-access-nnzxw\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.674260 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.674115    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/86970df8-d442-47d7-ba84-d7a7c84054bc-metrics-cert\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.674260 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.674159    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86970df8-d442-47d7-ba84-d7a7c84054bc-cert\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.674396 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.674250    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/86970df8-d442-47d7-ba84-d7a7c84054bc-manager-config\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.775125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.775085    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnzxw\" (UniqueName: \"kubernetes.io/projected/86970df8-d442-47d7-ba84-d7a7c84054bc-kube-api-access-nnzxw\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.775125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.775131    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/86970df8-d442-47d7-ba84-d7a7c84054bc-metrics-cert\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.775405 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.775198    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86970df8-d442-47d7-ba84-d7a7c84054bc-cert\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.775405 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.775300    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/86970df8-d442-47d7-ba84-d7a7c84054bc-manager-config\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.776563 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.776528    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/86970df8-d442-47d7-ba84-d7a7c84054bc-manager-config\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.779803 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.779771    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86970df8-d442-47d7-ba84-d7a7c84054bc-cert\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.782892 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.782867    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/86970df8-d442-47d7-ba84-d7a7c84054bc-metrics-cert\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.784233 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.784197    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnzxw\" (UniqueName: \"kubernetes.io/projected/86970df8-d442-47d7-ba84-d7a7c84054bc-kube-api-access-nnzxw\") pod \"lws-controller-manager-655586cd5f-w824w\" (UID: \"86970df8-d442-47d7-ba84-d7a7c84054bc\") " pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:51.929324 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:51.929229    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:52.201590 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:52.201507    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"]
Sep 08 14:22:52.203875 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:22:52.203844    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86970df8_d442_47d7_ba84_d7a7c84054bc.slice/crio-0ad334350b25c3d78259ba7cb81de8fb73f3bb95eed2d8c973483310b1086927 WatchSource:0}: Error finding container 0ad334350b25c3d78259ba7cb81de8fb73f3bb95eed2d8c973483310b1086927: Status 404 returned error can't find the container with id 0ad334350b25c3d78259ba7cb81de8fb73f3bb95eed2d8c973483310b1086927
Sep 08 14:22:52.894186 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:52.894144    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w" event={"ID":"86970df8-d442-47d7-ba84-d7a7c84054bc","Type":"ContainerStarted","Data":"0ad334350b25c3d78259ba7cb81de8fb73f3bb95eed2d8c973483310b1086927"}
Sep 08 14:22:53.757960 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:53.757874    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:53.849553 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:53.849489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:53.974129 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:53.974082    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:54.806031 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:54.805993    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:55.050060 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:55.050028    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:55.150259 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:55.150181    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:55.251715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:55.251680    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:55.910186 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:55.910149    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w" event={"ID":"86970df8-d442-47d7-ba84-d7a7c84054bc","Type":"ContainerStarted","Data":"6ce0c13828f7e5aec6c7bb87b1b8d42bdcba03317abc465c40f750da6db2281d"}
Sep 08 14:22:55.910376 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:55.910273    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:22:55.928395 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:55.928352    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w" podStartSLOduration=1.9504531539999999 podStartE2EDuration="4.928338606s" totalImagesPullingTime="2.977885452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:22:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:22:55.927972512 +0000 UTC m=+391.158903710" watchObservedRunningTime="2026-09-08 14:22:55.928338606 +0000 UTC m=+391.159269803"
Sep 08 14:22:57.181173 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:57.181138    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:57.181642 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:57.181238    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:57.185815 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:57.185791    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:57.923918 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:57.923887    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76785bcfcc-lnsvb"
Sep 08 14:22:58.083432 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:58.083402    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:58.184001 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:58.183917    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:58.285046 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:58.285009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:22:59.881908 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:22:59.881878    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kwhrf"
Sep 08 14:23:06.918741 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:23:06.918710    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-655586cd5f-w824w"
Sep 08 14:23:09.255115 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:23:09.255077    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:23:25.314911 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:23:25.314875    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:23:25.366348 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:23:25.366302    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:23:27.238847 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:23:27.238781    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:23:27.297822 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:23:27.297761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:23:45.440689 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:23:45.440653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:23:45.547074 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:23:45.547035    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:23:57.842285 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:23:57.842174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:23:58.647651 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:23:58.647620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:01.645499 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:01.645467    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:01.883939 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:01.883906    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:02.091630 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:02.091596    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:02.864272 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:02.864227    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:04.518026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:04.517994    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:04.642104 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:04.642071    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:05.858088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:05.858042    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:06.472061 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:06.472027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:08.068864 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:08.068829    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:08.165863 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:08.165830    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:10.864780 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:10.864747    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:10.964939 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:10.964903    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:11.068983 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:11.068952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:11.177079 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:11.177002    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:12.908201 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:12.908167    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"]
Sep 08 14:24:12.910911 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:12.910883    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:12.911603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:12.911589    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"
Sep 08 14:24:12.913958 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:12.913938    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 08 14:24:12.914054 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:12.913954    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 08 14:24:12.914335 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:12.914315    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-74m58\""
Sep 08 14:24:12.915093 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:12.915077    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 14:24:12.915173 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:12.915077    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 14:24:12.919113 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:12.919091    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"]
Sep 08 14:24:13.004503 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.004464    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b431fb06-c0af-416a-99e7-bde37580901e-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-zzqr9\" (UID: \"b431fb06-c0af-416a-99e7-bde37580901e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"
Sep 08 14:24:13.004707 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.004511    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b431fb06-c0af-416a-99e7-bde37580901e-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-zzqr9\" (UID: \"b431fb06-c0af-416a-99e7-bde37580901e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"
Sep 08 14:24:13.004707 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.004575    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vppq\" (UniqueName: \"kubernetes.io/projected/b431fb06-c0af-416a-99e7-bde37580901e-kube-api-access-2vppq\") pod \"kuadrant-console-plugin-598f86468b-zzqr9\" (UID: \"b431fb06-c0af-416a-99e7-bde37580901e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"
Sep 08 14:24:13.076713 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.076678    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:13.105006 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.104969    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b431fb06-c0af-416a-99e7-bde37580901e-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-zzqr9\" (UID: \"b431fb06-c0af-416a-99e7-bde37580901e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"
Sep 08 14:24:13.105158 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.105017    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b431fb06-c0af-416a-99e7-bde37580901e-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-zzqr9\" (UID: \"b431fb06-c0af-416a-99e7-bde37580901e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"
Sep 08 14:24:13.105158 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.105058    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vppq\" (UniqueName: \"kubernetes.io/projected/b431fb06-c0af-416a-99e7-bde37580901e-kube-api-access-2vppq\") pod \"kuadrant-console-plugin-598f86468b-zzqr9\" (UID: \"b431fb06-c0af-416a-99e7-bde37580901e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"
Sep 08 14:24:13.105763 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.105740    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b431fb06-c0af-416a-99e7-bde37580901e-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-zzqr9\" (UID: \"b431fb06-c0af-416a-99e7-bde37580901e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"
Sep 08 14:24:13.107575 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.107552    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b431fb06-c0af-416a-99e7-bde37580901e-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-zzqr9\" (UID: \"b431fb06-c0af-416a-99e7-bde37580901e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"
Sep 08 14:24:13.114060 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.114029    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vppq\" (UniqueName: \"kubernetes.io/projected/b431fb06-c0af-416a-99e7-bde37580901e-kube-api-access-2vppq\") pod \"kuadrant-console-plugin-598f86468b-zzqr9\" (UID: \"b431fb06-c0af-416a-99e7-bde37580901e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"
Sep 08 14:24:13.231922 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.231889    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"
Sep 08 14:24:13.502645 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:13.502472    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9"]
Sep 08 14:24:13.504919 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:24:13.504893    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb431fb06_c0af_416a_99e7_bde37580901e.slice/crio-4bc22b1f4e17a156f099c833d203b243229ad05c505fa69311c2ecd6bae952c0 WatchSource:0}: Error finding container 4bc22b1f4e17a156f099c833d203b243229ad05c505fa69311c2ecd6bae952c0: Status 404 returned error can't find the container with id 4bc22b1f4e17a156f099c833d203b243229ad05c505fa69311c2ecd6bae952c0
Sep 08 14:24:14.258822 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:14.258784    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9" event={"ID":"b431fb06-c0af-416a-99e7-bde37580901e","Type":"ContainerStarted","Data":"4bc22b1f4e17a156f099c833d203b243229ad05c505fa69311c2ecd6bae952c0"}
Sep 08 14:24:16.460189 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:16.460145    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:16.558748 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:16.558713    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:16.663651 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:16.663601    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:16.763398 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:16.763367    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:16.864955 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:16.864923    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:18.284306 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:18.284265    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9" event={"ID":"b431fb06-c0af-416a-99e7-bde37580901e","Type":"ContainerStarted","Data":"9b9e546659774142908846901eefb7b991deaa3da6e0ed18595d83fe479063fb"}
Sep 08 14:24:18.303534 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:18.303483    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-zzqr9" podStartSLOduration=1.738799601 podStartE2EDuration="6.303467426s" totalImagesPullingTime="4.564667825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:24:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:24:18.302566618 +0000 UTC m=+473.533497812" watchObservedRunningTime="2026-09-08 14:24:18.303467426 +0000 UTC m=+473.534398646"
Sep 08 14:24:18.836228 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:18.836184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:22.723835 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:22.723800    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:22.830715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:22.830669    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:28.124893 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:28.124858    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:28.223765 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:28.223735    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:29.830596 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:29.830562    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:29.931196 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:29.931162    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:33.282042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:33.281970    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:33.366402 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:33.366369    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:33.882349 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:33.882309    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:33.938678 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:33.938646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:39.701275 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:39.701239    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:39.801779 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:39.801744    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:40.280651 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:40.280614    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:40.348090 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:40.348054    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:40.883063 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:40.883027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:41.003034 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:41.002999    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:48.283777 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:48.283747    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:48.347159 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:48.347125    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:51.528734 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:51.528691    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:51.626530 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:51.626489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:53.161883 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:53.161847    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:24:53.478639 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:24:53.478542    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:03.980780 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:03.980738    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:04.033143 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:04.033111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:11.952599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:11.952542    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:12.061273 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.061234    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:12.162942 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.162906    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:12.353842 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.353803    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:25:12.377918 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.377880    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:25:12.378077 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.378004    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"
Sep 08 14:25:12.381055 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.381031    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 08 14:25:12.382570 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.382548    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:25:12.427813 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.427770    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4694b\" (UniqueName: \"kubernetes.io/projected/0f718b8a-b756-48e4-81f0-75041f5a3211-kube-api-access-4694b\") pod \"limitador-limitador-85d9db4d5d-pvltk\" (UID: \"0f718b8a-b756-48e4-81f0-75041f5a3211\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"
Sep 08 14:25:12.427813 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.427814    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0f718b8a-b756-48e4-81f0-75041f5a3211-config-file\") pod \"limitador-limitador-85d9db4d5d-pvltk\" (UID: \"0f718b8a-b756-48e4-81f0-75041f5a3211\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"
Sep 08 14:25:12.528557 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.528523    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4694b\" (UniqueName: \"kubernetes.io/projected/0f718b8a-b756-48e4-81f0-75041f5a3211-kube-api-access-4694b\") pod \"limitador-limitador-85d9db4d5d-pvltk\" (UID: \"0f718b8a-b756-48e4-81f0-75041f5a3211\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"
Sep 08 14:25:12.528557 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.528557    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0f718b8a-b756-48e4-81f0-75041f5a3211-config-file\") pod \"limitador-limitador-85d9db4d5d-pvltk\" (UID: \"0f718b8a-b756-48e4-81f0-75041f5a3211\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"
Sep 08 14:25:12.529176 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.529158    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0f718b8a-b756-48e4-81f0-75041f5a3211-config-file\") pod \"limitador-limitador-85d9db4d5d-pvltk\" (UID: \"0f718b8a-b756-48e4-81f0-75041f5a3211\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"
Sep 08 14:25:12.539831 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.539795    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4694b\" (UniqueName: \"kubernetes.io/projected/0f718b8a-b756-48e4-81f0-75041f5a3211-kube-api-access-4694b\") pod \"limitador-limitador-85d9db4d5d-pvltk\" (UID: \"0f718b8a-b756-48e4-81f0-75041f5a3211\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"
Sep 08 14:25:12.688539 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.688427    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"
Sep 08 14:25:12.824899 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.824856    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:12.919621 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.919583    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:12.959216 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:12.959176    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:25:12.961415 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:25:12.961385    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f718b8a_b756_48e4_81f0_75041f5a3211.slice/crio-6cf0eb16ec075f53fcf23c29e02c742143ae840391ba61900c02dfb5d706e4a1 WatchSource:0}: Error finding container 6cf0eb16ec075f53fcf23c29e02c742143ae840391ba61900c02dfb5d706e4a1: Status 404 returned error can't find the container with id 6cf0eb16ec075f53fcf23c29e02c742143ae840391ba61900c02dfb5d706e4a1
Sep 08 14:25:13.529710 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:13.529673    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk" event={"ID":"0f718b8a-b756-48e4-81f0-75041f5a3211","Type":"ContainerStarted","Data":"6cf0eb16ec075f53fcf23c29e02c742143ae840391ba61900c02dfb5d706e4a1"}
Sep 08 14:25:15.541278 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:15.541237    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk" event={"ID":"0f718b8a-b756-48e4-81f0-75041f5a3211","Type":"ContainerStarted","Data":"4168810c01546fd43424a279f34298c88e389f4db6273f0ecad3d71d2060c2ea"}
Sep 08 14:25:15.541710 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:15.541359    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"
Sep 08 14:25:15.559280 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:15.559220    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk" podStartSLOduration=1.338979738 podStartE2EDuration="3.559188202s" totalImagesPullingTime="2.220208464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:25:15.557057679 +0000 UTC m=+530.787988875" watchObservedRunningTime="2026-09-08 14:25:15.559188202 +0000 UTC m=+530.790119400"
Sep 08 14:25:26.547089 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:26.546994    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"
Sep 08 14:25:31.428360 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:31.428309    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:31.527654 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:31.527597    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:31.940637 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:31.940600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:44.458462 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:44.458423    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:45.031154 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:45.031117    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:52.734698 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:52.734656    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:52.834809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:52.834777    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:55.345995 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:55.345953    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:55.446808 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:55.446767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:55.536982 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:55.536939    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:55.596619 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:55.596525    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:55.695779 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:55.695746    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:58.956737 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:58.956702    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:25:59.064261 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:25:59.064227    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:26:06.949031 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:26:06.948996    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:26:14.474026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:26:14.473988    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:26:31.598298 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:26:31.598262    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:26:31.699127 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:26:31.699091    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:26:33.088474 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:26:33.088423    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:26:33.864047 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:26:33.864007    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:26:36.371795 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:26:36.371756    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:26:41.113224 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:26:41.113184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:26:47.539295 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:26:47.539231    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:26:49.589577 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:26:49.589544    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:26:49.691854 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:26:49.691804    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:27:04.955756 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:04.955721    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"]
Sep 08 14:27:04.961798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:04.961764    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:04.967040 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:04.966336    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-68fbj\""
Sep 08 14:27:04.967040 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:04.966347    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 14:27:04.967040 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:04.966429    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 14:27:04.967040 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:04.966762    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 08 14:27:04.969366 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:04.969335    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"]
Sep 08 14:27:04.994481 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:04.994456    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:27:05.027714 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.027684    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.027830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.027736    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.027830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.027795    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.027905 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.027838    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.027905 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.027856    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.027905 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.027873    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpvcl\" (UniqueName: \"kubernetes.io/projected/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-kube-api-access-qpvcl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.128883 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.128848    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.128883 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.128887    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.129084 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.128910    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.129084 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.128934    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpvcl\" (UniqueName: \"kubernetes.io/projected/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-kube-api-access-qpvcl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.129181 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.129096    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.129181 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.129155    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.129346 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.129328    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.129410 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.129385    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.129483 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.129464    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.131031 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.131012    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.131436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.131419    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.138518 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.138499    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpvcl\" (UniqueName: \"kubernetes.io/projected/9d16d9e0-a2fe-4d90-810d-22a85e7e3b85-kube-api-access-qpvcl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn\" (UID: \"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.276554 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.276504    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:05.465145 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.465108    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:27:05.549651 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:05.549626    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"]
Sep 08 14:27:05.552248 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:27:05.552219    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d16d9e0_a2fe_4d90_810d_22a85e7e3b85.slice/crio-f4f2d32dbf4a8113390e663a013508e41477796d8d8326c4a0cfbd63e57369db WatchSource:0}: Error finding container f4f2d32dbf4a8113390e663a013508e41477796d8d8326c4a0cfbd63e57369db: Status 404 returned error can't find the container with id f4f2d32dbf4a8113390e663a013508e41477796d8d8326c4a0cfbd63e57369db
Sep 08 14:27:06.034543 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:06.034505    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn" event={"ID":"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85","Type":"ContainerStarted","Data":"f4f2d32dbf4a8113390e663a013508e41477796d8d8326c4a0cfbd63e57369db"}
Sep 08 14:27:07.040243 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:07.040187    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn" event={"ID":"9d16d9e0-a2fe-4d90-810d-22a85e7e3b85","Type":"ContainerStarted","Data":"95aa51e114b7ff8b127f3895c50ec381fd35b4aae78fe78e05e810113a11ad96"}
Sep 08 14:27:07.064454 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:07.064396    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn" podStartSLOduration=1.915897966 podStartE2EDuration="3.064378494s" totalImagesPullingTime="1.148480528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:27:07.058545964 +0000 UTC m=+642.289477160" watchObservedRunningTime="2026-09-08 14:27:07.064378494 +0000 UTC m=+642.295309690"
Sep 08 14:27:09.179107 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:09.179059    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:27:09.852656 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:09.852621    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:27:15.276809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:15.276768    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:15.277199 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:15.276841    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:15.289068 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:15.289036    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:16.090876 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:16.090841    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn"
Sep 08 14:27:18.949508 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:18.949475    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:27:19.392271 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:19.392235    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:27:19.493075 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:19.493040    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:27:19.856826 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:19.856787    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:27:38.265091 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.265059    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"]
Sep 08 14:27:38.268637 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.268620    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.271314 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.271291    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 08 14:27:38.276891 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.276867    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"]
Sep 08 14:27:38.297934 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.297905    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:27:38.322223 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.322183    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.322319 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.322268    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.322319 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.322294    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.322399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.322331    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78f55cf2-a241-4538-a5f4-ef105e68cdd2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.322399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.322348    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6bt8\" (UniqueName: \"kubernetes.io/projected/78f55cf2-a241-4538-a5f4-ef105e68cdd2-kube-api-access-w6bt8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.322399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.322387    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.423198 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.423162    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.423379 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.423245    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.423379 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.423264    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.423379 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.423286    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78f55cf2-a241-4538-a5f4-ef105e68cdd2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.423379 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.423302    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6bt8\" (UniqueName: \"kubernetes.io/projected/78f55cf2-a241-4538-a5f4-ef105e68cdd2-kube-api-access-w6bt8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.423379 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.423345    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.423686 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.423663    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.423747 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.423684    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.423747 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.423694    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.425683 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.425659    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78f55cf2-a241-4538-a5f4-ef105e68cdd2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.425959 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.425937    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78f55cf2-a241-4538-a5f4-ef105e68cdd2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.432325 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.432304    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6bt8\" (UniqueName: \"kubernetes.io/projected/78f55cf2-a241-4538-a5f4-ef105e68cdd2-kube-api-access-w6bt8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74k5g\" (UID: \"78f55cf2-a241-4538-a5f4-ef105e68cdd2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.579537 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.579440    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:38.850736 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.850711    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"]
Sep 08 14:27:38.853476 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:27:38.853447    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f55cf2_a241_4538_a5f4_ef105e68cdd2.slice/crio-4fbd68a53b72ce57b3c940acc133ee4477ce43ae082de7c54bd03300bd4b754a WatchSource:0}: Error finding container 4fbd68a53b72ce57b3c940acc133ee4477ce43ae082de7c54bd03300bd4b754a: Status 404 returned error can't find the container with id 4fbd68a53b72ce57b3c940acc133ee4477ce43ae082de7c54bd03300bd4b754a
Sep 08 14:27:38.854862 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:38.854842    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:27:39.184733 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:39.184655    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g" event={"ID":"78f55cf2-a241-4538-a5f4-ef105e68cdd2","Type":"ContainerStarted","Data":"e2ae877bf6903575e3e899acd8805e37bc488c8275e99fdf2febb7337af76ee5"}
Sep 08 14:27:39.184733 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:39.184707    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g" event={"ID":"78f55cf2-a241-4538-a5f4-ef105e68cdd2","Type":"ContainerStarted","Data":"4fbd68a53b72ce57b3c940acc133ee4477ce43ae082de7c54bd03300bd4b754a"}
Sep 08 14:27:39.203311 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:39.203268    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g" podStartSLOduration=1.026890428 podStartE2EDuration="1.20325224s" totalImagesPullingTime="176.361812ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:27:39.201099287 +0000 UTC m=+674.432030485" watchObservedRunningTime="2026-09-08 14:27:39.20325224 +0000 UTC m=+674.434183437"
Sep 08 14:27:39.550061 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:39.550026    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:27:48.580220 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:48.580164    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:48.580738 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:48.580283    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:48.593605 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:48.593572    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:49.241589 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:49.241555    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74k5g"
Sep 08 14:27:55.659794 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:55.659758    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:27:56.066380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:27:56.066347    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:28:09.057011 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.056979    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"]
Sep 08 14:28:09.060583 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.060567    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.063345 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.063304    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 08 14:28:09.071065 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.071045    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"]
Sep 08 14:28:09.090466 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.090427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:28:09.096526 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.096507    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.096603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.096541    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/381d79dc-fc22-415e-862c-9b252e3f2484-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.096603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.096577    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.096702 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.096689    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkc99\" (UniqueName: \"kubernetes.io/projected/381d79dc-fc22-415e-862c-9b252e3f2484-kube-api-access-nkc99\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.096750 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.096711    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.096750 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.096724    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.197607 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.197579    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkc99\" (UniqueName: \"kubernetes.io/projected/381d79dc-fc22-415e-862c-9b252e3f2484-kube-api-access-nkc99\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.197703 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.197610    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.197703 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.197627    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.197703 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.197645    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.197703 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.197667    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/381d79dc-fc22-415e-862c-9b252e3f2484-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.197877 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.197810    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.198110 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.198081    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.198110 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.198100    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.198297 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.198158    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.199921 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.199892    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/381d79dc-fc22-415e-862c-9b252e3f2484-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.200201 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.200180    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/381d79dc-fc22-415e-862c-9b252e3f2484-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.208106 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.208088    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkc99\" (UniqueName: \"kubernetes.io/projected/381d79dc-fc22-415e-862c-9b252e3f2484-kube-api-access-nkc99\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pjpg9\" (UID: \"381d79dc-fc22-415e-862c-9b252e3f2484\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.371511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.371436    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:09.646251 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:09.646223    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"]
Sep 08 14:28:09.648121 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:28:09.648095    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod381d79dc_fc22_415e_862c_9b252e3f2484.slice/crio-0650bdfe230489647f61ad782bc8beda4302ef4071322066a8aab30ee0998ff0 WatchSource:0}: Error finding container 0650bdfe230489647f61ad782bc8beda4302ef4071322066a8aab30ee0998ff0: Status 404 returned error can't find the container with id 0650bdfe230489647f61ad782bc8beda4302ef4071322066a8aab30ee0998ff0
Sep 08 14:28:10.326268 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:10.326233    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9" event={"ID":"381d79dc-fc22-415e-862c-9b252e3f2484","Type":"ContainerStarted","Data":"56e5208d1e5e7ec6151d5db41d2f83850ad6bc7bb9a2e907bb1c45b65583bf69"}
Sep 08 14:28:10.326619 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:10.326285    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9" event={"ID":"381d79dc-fc22-415e-862c-9b252e3f2484","Type":"ContainerStarted","Data":"0650bdfe230489647f61ad782bc8beda4302ef4071322066a8aab30ee0998ff0"}
Sep 08 14:28:10.344775 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:10.344724    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9" podStartSLOduration=1.144688075 podStartE2EDuration="1.344711084s" totalImagesPullingTime="200.023009ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:28:10.342925752 +0000 UTC m=+705.573856947" watchObservedRunningTime="2026-09-08 14:28:10.344711084 +0000 UTC m=+705.575642283"
Sep 08 14:28:10.755108 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:10.755069    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:28:17.238432 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.238399    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"]
Sep 08 14:28:17.242069 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.242046    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.244418 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.244396    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 08 14:28:17.253120 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.253094    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"]
Sep 08 14:28:17.265434 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.265409    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:28:17.371562 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.371530    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.371714 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.371596    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.371714 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.371632    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.371714 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.371651    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.371820 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.371712    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmplw\" (UniqueName: \"kubernetes.io/projected/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-kube-api-access-dmplw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.371820 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.371799    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.472930 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.472893    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.472930 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.472934    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.473139 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.472960    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.473139 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.473000    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmplw\" (UniqueName: \"kubernetes.io/projected/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-kube-api-access-dmplw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.473139 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.473059    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.473139 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.473090    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.473392 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.473367    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.473455 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.473424    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.473455 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.473441    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.475258 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.475236    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.475622 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.475605    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.482162 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.482138    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmplw\" (UniqueName: \"kubernetes.io/projected/dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e-kube-api-access-dmplw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grbh6\" (UID: \"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.552692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.552617    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:17.822862 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:17.822834    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"]
Sep 08 14:28:17.824677 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:28:17.824652    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd4c4ed5_9c56_4596_91fc_97e9a6e5f38e.slice/crio-f3b6213000f15cb3569029ec3e01a165f71c85d7392f1c9badab75ace4084267 WatchSource:0}: Error finding container f3b6213000f15cb3569029ec3e01a165f71c85d7392f1c9badab75ace4084267: Status 404 returned error can't find the container with id f3b6213000f15cb3569029ec3e01a165f71c85d7392f1c9badab75ace4084267
Sep 08 14:28:18.362108 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:18.362018    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6" event={"ID":"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e","Type":"ContainerStarted","Data":"07ad587e6711bb8da4e7b0553410d497f46fec305d15c9a37b84b52343c5fd29"}
Sep 08 14:28:18.362108 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:18.362065    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6" event={"ID":"dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e","Type":"ContainerStarted","Data":"f3b6213000f15cb3569029ec3e01a165f71c85d7392f1c9badab75ace4084267"}
Sep 08 14:28:18.380744 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:18.380696    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6" podStartSLOduration=1.164162078 podStartE2EDuration="1.380681006s" totalImagesPullingTime="216.518928ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:28:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:28:18.379316619 +0000 UTC m=+713.610247816" watchObservedRunningTime="2026-09-08 14:28:18.380681006 +0000 UTC m=+713.611612246"
Sep 08 14:28:19.372422 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:19.372388    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:19.372787 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:19.372444    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:19.385306 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:19.385282    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:19.455068 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:19.455037    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:28:20.382196 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:20.382163    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pjpg9"
Sep 08 14:28:27.552946 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:27.552914    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:27.553338 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:27.553133    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:27.565700 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:27.565676    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:28:28.416408 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:28:28.416378    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grbh6"
Sep 08 14:29:42.087505 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:29:42.087462    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:29:42.187439 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:29:42.187401    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:29:43.254511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:29:43.254469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:29:43.348616 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:29:43.348581    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:29:58.787813 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:29:58.787733    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:29:58.904330 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:29:58.904303    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:29:59.145991 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:29:59.145916    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:00.846544 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:00.846506    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:00.907067 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:00.907030    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:02.386566 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:02.386526    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:02.523493 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:02.523461    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:02.862694 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:02.862659    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:30:03.541049 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:03.541013    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:03.642391 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:03.642356    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:03.861678 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:03.861596    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:04.161865 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:04.161795    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:04.259802 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:04.259771    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:07.158943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:07.158912    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:07.243241 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:07.243170    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:07.386809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:07.386776    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:07.698720 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:07.698691    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:07.800902 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:07.800863    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:09.869425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:09.869389    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:09.961432 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:09.961400    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:11.073523 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.073487    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:11.136084 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.136054    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:11.239481 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.239447    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:11.612744 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.612713    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:11.710885 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.710850    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:11.742973 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.742941    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"]
Sep 08 14:30:11.746419 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.746404    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:11.748689 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.748665    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-32df8e95\""
Sep 08 14:30:11.748828 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.748746    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-32df8e95-dockercfg-6cx2q\""
Sep 08 14:30:11.758515 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.758492    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"]
Sep 08 14:30:11.779905 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.779883    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"]
Sep 08 14:30:11.783283 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.783265    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:11.799421 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.799389    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"]
Sep 08 14:30:11.900471 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.900396    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bc8m\" (UniqueName: \"kubernetes.io/projected/6e987115-a47a-46d4-ac46-938a75940b54-kube-api-access-7bc8m\") pod \"payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz\" (UID: \"6e987115-a47a-46d4-ac46-938a75940b54\") " pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:11.900471 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.900440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e987115-a47a-46d4-ac46-938a75940b54-plugins-config-volume\") pod \"payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz\" (UID: \"6e987115-a47a-46d4-ac46-938a75940b54\") " pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:11.900637 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.900527    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d523560a-6b69-4633-bcda-823a901a0c94-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv\" (UID: \"d523560a-6b69-4633-bcda-823a901a0c94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:11.900637 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:11.900559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpd5v\" (UniqueName: \"kubernetes.io/projected/d523560a-6b69-4633-bcda-823a901a0c94-kube-api-access-wpd5v\") pod \"payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv\" (UID: \"d523560a-6b69-4633-bcda-823a901a0c94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:12.001289 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.001251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bc8m\" (UniqueName: \"kubernetes.io/projected/6e987115-a47a-46d4-ac46-938a75940b54-kube-api-access-7bc8m\") pod \"payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz\" (UID: \"6e987115-a47a-46d4-ac46-938a75940b54\") " pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:12.001452 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.001299    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e987115-a47a-46d4-ac46-938a75940b54-plugins-config-volume\") pod \"payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz\" (UID: \"6e987115-a47a-46d4-ac46-938a75940b54\") " pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:12.001452 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.001349    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d523560a-6b69-4633-bcda-823a901a0c94-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv\" (UID: \"d523560a-6b69-4633-bcda-823a901a0c94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:12.001452 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.001368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpd5v\" (UniqueName: \"kubernetes.io/projected/d523560a-6b69-4633-bcda-823a901a0c94-kube-api-access-wpd5v\") pod \"payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv\" (UID: \"d523560a-6b69-4633-bcda-823a901a0c94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:12.002076 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.002056    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e987115-a47a-46d4-ac46-938a75940b54-plugins-config-volume\") pod \"payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz\" (UID: \"6e987115-a47a-46d4-ac46-938a75940b54\") " pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:12.002145 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.002125    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d523560a-6b69-4633-bcda-823a901a0c94-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv\" (UID: \"d523560a-6b69-4633-bcda-823a901a0c94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:12.010513 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.010486    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpd5v\" (UniqueName: \"kubernetes.io/projected/d523560a-6b69-4633-bcda-823a901a0c94-kube-api-access-wpd5v\") pod \"payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv\" (UID: \"d523560a-6b69-4633-bcda-823a901a0c94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:12.010513 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.010503    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bc8m\" (UniqueName: \"kubernetes.io/projected/6e987115-a47a-46d4-ac46-938a75940b54-kube-api-access-7bc8m\") pod \"payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz\" (UID: \"6e987115-a47a-46d4-ac46-938a75940b54\") " pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:12.056051 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.056022    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:12.100007 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.099975    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:12.325095 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.325071    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"]
Sep 08 14:30:12.326879 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:30:12.326851    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e987115_a47a_46d4_ac46_938a75940b54.slice/crio-3c782c0bd356e199b15c8a225adefa31df33904186e2c9ed835bc1e892dadec2 WatchSource:0}: Error finding container 3c782c0bd356e199b15c8a225adefa31df33904186e2c9ed835bc1e892dadec2: Status 404 returned error can't find the container with id 3c782c0bd356e199b15c8a225adefa31df33904186e2c9ed835bc1e892dadec2
Sep 08 14:30:12.385864 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.385833    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"]
Sep 08 14:30:12.387404 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:30:12.387380    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd523560a_6b69_4633_bcda_823a901a0c94.slice/crio-cae0dbe94581c86e649eac7d872e30ea76774dba968d8cf453fe53d856090da9 WatchSource:0}: Error finding container cae0dbe94581c86e649eac7d872e30ea76774dba968d8cf453fe53d856090da9: Status 404 returned error can't find the container with id cae0dbe94581c86e649eac7d872e30ea76774dba968d8cf453fe53d856090da9
Sep 08 14:30:12.887917 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.887874    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz" event={"ID":"6e987115-a47a-46d4-ac46-938a75940b54","Type":"ContainerStarted","Data":"3c782c0bd356e199b15c8a225adefa31df33904186e2c9ed835bc1e892dadec2"}
Sep 08 14:30:12.889426 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:12.889399    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv" event={"ID":"d523560a-6b69-4633-bcda-823a901a0c94","Type":"ContainerStarted","Data":"cae0dbe94581c86e649eac7d872e30ea76774dba968d8cf453fe53d856090da9"}
Sep 08 14:30:13.471718 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:13.471681    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7907-dc4f1d"]
Sep 08 14:30:13.475802 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:13.475780    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7907-dc4f1d"
Sep 08 14:30:13.478966 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:13.478942    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qcp6n\""
Sep 08 14:30:13.480988 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:13.480942    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7907-dc4f1d"]
Sep 08 14:30:13.618513 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:13.618474    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vkz8\" (UniqueName: \"kubernetes.io/projected/7470aa0f-e000-420e-a746-09124b5ceb0e-kube-api-access-6vkz8\") pod \"test-curl-7907-dc4f1d\" (UID: \"7470aa0f-e000-420e-a746-09124b5ceb0e\") " pod="opendatahub/test-curl-7907-dc4f1d"
Sep 08 14:30:13.719182 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:13.719141    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vkz8\" (UniqueName: \"kubernetes.io/projected/7470aa0f-e000-420e-a746-09124b5ceb0e-kube-api-access-6vkz8\") pod \"test-curl-7907-dc4f1d\" (UID: \"7470aa0f-e000-420e-a746-09124b5ceb0e\") " pod="opendatahub/test-curl-7907-dc4f1d"
Sep 08 14:30:13.728532 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:13.728468    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vkz8\" (UniqueName: \"kubernetes.io/projected/7470aa0f-e000-420e-a746-09124b5ceb0e-kube-api-access-6vkz8\") pod \"test-curl-7907-dc4f1d\" (UID: \"7470aa0f-e000-420e-a746-09124b5ceb0e\") " pod="opendatahub/test-curl-7907-dc4f1d"
Sep 08 14:30:13.790553 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:13.790524    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7907-dc4f1d"
Sep 08 14:30:14.077879 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:14.077851    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7907-dc4f1d"]
Sep 08 14:30:14.537096 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:30:14.537059    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7470aa0f_e000_420e_a746_09124b5ceb0e.slice/crio-51b11169a64310b486276ca8b3dbcc37ae9f8e1e41c7f75099a70baa517febe9 WatchSource:0}: Error finding container 51b11169a64310b486276ca8b3dbcc37ae9f8e1e41c7f75099a70baa517febe9: Status 404 returned error can't find the container with id 51b11169a64310b486276ca8b3dbcc37ae9f8e1e41c7f75099a70baa517febe9
Sep 08 14:30:14.901440 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:14.901410    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7907-dc4f1d" event={"ID":"7470aa0f-e000-420e-a746-09124b5ceb0e","Type":"ContainerStarted","Data":"51b11169a64310b486276ca8b3dbcc37ae9f8e1e41c7f75099a70baa517febe9"}
Sep 08 14:30:15.559435 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:15.559394    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:30:15.908099 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:15.908002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz" event={"ID":"6e987115-a47a-46d4-ac46-938a75940b54","Type":"ContainerStarted","Data":"578d30bc9e1b4e0320bc53871ab08bf89daf109d11b9a6507a0835ad82f34570"}
Sep 08 14:30:15.908099 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:15.908067    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:15.909665 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:15.909633    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv" event={"ID":"d523560a-6b69-4633-bcda-823a901a0c94","Type":"ContainerStarted","Data":"a2821fa146393a0b4f6721cb0474036aadc91736fd41a48290f261d5302b8e68"}
Sep 08 14:30:15.909778 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:15.909686    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:15.926394 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:15.926350    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz" podStartSLOduration=2.342355535 podStartE2EDuration="4.926337357s" totalImagesPullingTime="2.583981822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:30:15.923620109 +0000 UTC m=+831.154551305" watchObservedRunningTime="2026-09-08 14:30:15.926337357 +0000 UTC m=+831.157268555"
Sep 08 14:30:15.941164 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:15.941116    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv" podStartSLOduration=2.42444194 podStartE2EDuration="4.941101204s" totalImagesPullingTime="2.516659264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:30:15.940724985 +0000 UTC m=+831.171656179" watchObservedRunningTime="2026-09-08 14:30:15.941101204 +0000 UTC m=+831.172032400"
Sep 08 14:30:17.922528 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:17.922490    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7907-dc4f1d" event={"ID":"7470aa0f-e000-420e-a746-09124b5ceb0e","Type":"ContainerStarted","Data":"932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63"}
Sep 08 14:30:17.936539 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:17.936493    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7907-dc4f1d" podStartSLOduration=2.4630135490000002 podStartE2EDuration="4.93647865s" totalImagesPullingTime="2.473465101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:30:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:30:17.935841704 +0000 UTC m=+833.166772901" watchObservedRunningTime="2026-09-08 14:30:17.93647865 +0000 UTC m=+833.167409846"
Sep 08 14:30:18.102720 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:18.102687    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:18.245652 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:18.245591    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:18.347431 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:18.347396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:18.445018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:18.444980    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:18.629162 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:18.629069    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7907-dc4f1d"]
Sep 08 14:30:18.631436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:18.631408    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7907-dc4f1d"]
Sep 08 14:30:18.928991 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:30:18.928911    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7907-dc4f1d" container="test-curl-7907-dc4f1d"
Sep 08 14:30:19.172889 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:19.172855    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:19.530869 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:19.530836    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:19.933363 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:19.933198    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7907-dc4f1d" podUID="7470aa0f-e000-420e-a746-09124b5ceb0e" containerName="test-curl-7907-dc4f1d" containerID="cri-o://932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63" gracePeriod=2
Sep 08 14:30:20.183709 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:20.183629    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:20.283532 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:20.283500    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:20.384241 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:20.384191    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:20.483589 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:20.483554    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:21.179905 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:21.179875    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:22.110912 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.110890    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7907-dc4f1d"
Sep 08 14:30:22.196060 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.195996    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vkz8\" (UniqueName: \"kubernetes.io/projected/7470aa0f-e000-420e-a746-09124b5ceb0e-kube-api-access-6vkz8\") pod \"7470aa0f-e000-420e-a746-09124b5ceb0e\" (UID: \"7470aa0f-e000-420e-a746-09124b5ceb0e\") "
Sep 08 14:30:22.198062 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.198037    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7470aa0f-e000-420e-a746-09124b5ceb0e-kube-api-access-6vkz8" (OuterVolumeSpecName: "kube-api-access-6vkz8") pod "7470aa0f-e000-420e-a746-09124b5ceb0e" (UID: "7470aa0f-e000-420e-a746-09124b5ceb0e"). InnerVolumeSpecName "kube-api-access-6vkz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:30:22.297387 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.297352    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vkz8\" (UniqueName: \"kubernetes.io/projected/7470aa0f-e000-420e-a746-09124b5ceb0e-kube-api-access-6vkz8\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:30:22.548808 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.548774    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:22.645796 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.645756    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:22.823650 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.823570    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:22.950220 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.950180    2558 generic.go:356] "Generic (PLEG): container finished" podID="7470aa0f-e000-420e-a746-09124b5ceb0e" containerID="932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63" exitCode=137
Sep 08 14:30:22.950384 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.950250    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7907-dc4f1d"
Sep 08 14:30:22.950384 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.950251    2558 scope.go:117] "RemoveContainer" containerID="932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63"
Sep 08 14:30:22.959357 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.959331    2558 scope.go:117] "RemoveContainer" containerID="932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63"
Sep 08 14:30:22.959621 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:30:22.959600    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63\": container with ID starting with 932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63 not found: ID does not exist" containerID="932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63"
Sep 08 14:30:22.959688 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:22.959633    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63"} err="failed to get container status \"932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63\": rpc error: code = NotFound desc = could not find container \"932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63\": container with ID starting with 932fb884ce7344f1708d7cd960910722aee8516564aeaf1f81e23d406ecf9f63 not found: ID does not exist"
Sep 08 14:30:22.960768 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:30:22.960740    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7907-dc4f1d\" is forbidden: User \"system:node:ip-10-0-131-22.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-22.ec2.internal' and this object" podUID="7470aa0f-e000-420e-a746-09124b5ceb0e" pod="opendatahub/test-curl-7907-dc4f1d"
Sep 08 14:30:23.286721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:23.286685    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7470aa0f-e000-420e-a746-09124b5ceb0e" path="/var/lib/kubelet/pods/7470aa0f-e000-420e-a746-09124b5ceb0e/volumes"
Sep 08 14:30:23.707941 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:23.707860    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:23.810818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:23.810789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:24.112879 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.112845    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:24.218839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.218802    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:24.315788 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.315759    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:24.330658 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.330634    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"]
Sep 08 14:30:24.331053 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.331038    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7470aa0f-e000-420e-a746-09124b5ceb0e" containerName="test-curl-7907-dc4f1d"
Sep 08 14:30:24.331096 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.331055    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7470aa0f-e000-420e-a746-09124b5ceb0e" containerName="test-curl-7907-dc4f1d"
Sep 08 14:30:24.331132 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.331114    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7470aa0f-e000-420e-a746-09124b5ceb0e" containerName="test-curl-7907-dc4f1d"
Sep 08 14:30:24.335420 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.335404    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:30:24.338481 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.338436    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-47d144-dockercfg-nxghc\""
Sep 08 14:30:24.338481 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.338455    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-47d144\""
Sep 08 14:30:24.342475 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.342309    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"]
Sep 08 14:30:24.413624 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.413554    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln\" (UID: \"eafebe59-c6ca-4321-a6f2-8f8da114c2b7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:30:24.413624 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.413605    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grpks\" (UniqueName: \"kubernetes.io/projected/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-kube-api-access-grpks\") pod \"payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln\" (UID: \"eafebe59-c6ca-4321-a6f2-8f8da114c2b7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:30:24.514333 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.514300    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln\" (UID: \"eafebe59-c6ca-4321-a6f2-8f8da114c2b7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:30:24.514486 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.514351    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grpks\" (UniqueName: \"kubernetes.io/projected/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-kube-api-access-grpks\") pod \"payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln\" (UID: \"eafebe59-c6ca-4321-a6f2-8f8da114c2b7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:30:24.515036 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.515017    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln\" (UID: \"eafebe59-c6ca-4321-a6f2-8f8da114c2b7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:30:24.523266 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.523236    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grpks\" (UniqueName: \"kubernetes.io/projected/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-kube-api-access-grpks\") pod \"payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln\" (UID: \"eafebe59-c6ca-4321-a6f2-8f8da114c2b7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:30:24.648690 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.648659    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:30:24.752543 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.752514    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:24.918671 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.918644    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"]
Sep 08 14:30:24.920738 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:30:24.920713    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeafebe59_c6ca_4321_a6f2_8f8da114c2b7.slice/crio-d7836a95beb3a83bf0677d3bef1fce5feec90140e198a663baed6c55a46b2154 WatchSource:0}: Error finding container d7836a95beb3a83bf0677d3bef1fce5feec90140e198a663baed6c55a46b2154: Status 404 returned error can't find the container with id d7836a95beb3a83bf0677d3bef1fce5feec90140e198a663baed6c55a46b2154
Sep 08 14:30:24.962383 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:24.962326    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln" event={"ID":"eafebe59-c6ca-4321-a6f2-8f8da114c2b7","Type":"ContainerStarted","Data":"d7836a95beb3a83bf0677d3bef1fce5feec90140e198a663baed6c55a46b2154"}
Sep 08 14:30:25.584849 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:25.584765    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:25.970610 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:25.970521    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln" event={"ID":"eafebe59-c6ca-4321-a6f2-8f8da114c2b7","Type":"ContainerStarted","Data":"49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340"}
Sep 08 14:30:25.970839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:25.970822    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:30:25.988032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:25.987982    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln" podStartSLOduration=1.987967877 podStartE2EDuration="1.987967877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:30:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:30:25.986652087 +0000 UTC m=+841.217583286" watchObservedRunningTime="2026-09-08 14:30:25.987967877 +0000 UTC m=+841.218899073"
Sep 08 14:30:26.405730 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:26.405698    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:26.917967 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:26.917928    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:26.918385 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:26.918168    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:27.375920 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:27.375875    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:27.710662 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:27.710576    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:27.915798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:27.915764    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:28.028271 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:28.028225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:30.518880 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:30.518851    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:30.582509 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:30.582482    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:31.576553 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:31.576520    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:31.667128 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:31.667097    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:32.736022 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:32.735986    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:32.836355 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:32.836299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:32.953819 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:32.953759    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:33.359520 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.359481    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:33.442566 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.442538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:33.443604 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.443577    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"]
Sep 08 14:30:33.448539 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.448520    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:30:33.451445 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.451417    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-2a2de3-dockercfg-mnmjj\""
Sep 08 14:30:33.451663 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.451646    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-2a2de3\""
Sep 08 14:30:33.458431 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.458407    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"]
Sep 08 14:30:33.460144 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.460124    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"]
Sep 08 14:30:33.463463 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.463441    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:30:33.479119 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.479096    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"]
Sep 08 14:30:33.487788 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.487762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fm98\" (UniqueName: \"kubernetes.io/projected/fd458ea6-161b-489e-b07d-905be3ef51eb-kube-api-access-6fm98\") pod \"payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz\" (UID: \"fd458ea6-161b-489e-b07d-905be3ef51eb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:30:33.487899 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.487811    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd458ea6-161b-489e-b07d-905be3ef51eb-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz\" (UID: \"fd458ea6-161b-489e-b07d-905be3ef51eb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:30:33.487954 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.487905    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs857\" (UniqueName: \"kubernetes.io/projected/b66ead7b-a5c7-48c7-87ed-502eb4507230-kube-api-access-bs857\") pod \"payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s\" (UID: \"b66ead7b-a5c7-48c7-87ed-502eb4507230\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:30:33.487993 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.487954    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b66ead7b-a5c7-48c7-87ed-502eb4507230-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s\" (UID: \"b66ead7b-a5c7-48c7-87ed-502eb4507230\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:30:33.588434 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.588402    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs857\" (UniqueName: \"kubernetes.io/projected/b66ead7b-a5c7-48c7-87ed-502eb4507230-kube-api-access-bs857\") pod \"payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s\" (UID: \"b66ead7b-a5c7-48c7-87ed-502eb4507230\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:30:33.588588 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.588447    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b66ead7b-a5c7-48c7-87ed-502eb4507230-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s\" (UID: \"b66ead7b-a5c7-48c7-87ed-502eb4507230\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:30:33.588588 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.588503    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fm98\" (UniqueName: \"kubernetes.io/projected/fd458ea6-161b-489e-b07d-905be3ef51eb-kube-api-access-6fm98\") pod \"payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz\" (UID: \"fd458ea6-161b-489e-b07d-905be3ef51eb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:30:33.588588 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.588523    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd458ea6-161b-489e-b07d-905be3ef51eb-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz\" (UID: \"fd458ea6-161b-489e-b07d-905be3ef51eb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:30:33.589217 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.589174    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd458ea6-161b-489e-b07d-905be3ef51eb-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz\" (UID: \"fd458ea6-161b-489e-b07d-905be3ef51eb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:30:33.589329 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.589282    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b66ead7b-a5c7-48c7-87ed-502eb4507230-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s\" (UID: \"b66ead7b-a5c7-48c7-87ed-502eb4507230\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:30:33.601199 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.601176    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs857\" (UniqueName: \"kubernetes.io/projected/b66ead7b-a5c7-48c7-87ed-502eb4507230-kube-api-access-bs857\") pod \"payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s\" (UID: \"b66ead7b-a5c7-48c7-87ed-502eb4507230\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:30:33.602147 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.602112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fm98\" (UniqueName: \"kubernetes.io/projected/fd458ea6-161b-489e-b07d-905be3ef51eb-kube-api-access-6fm98\") pod \"payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz\" (UID: \"fd458ea6-161b-489e-b07d-905be3ef51eb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:30:33.760908 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.760881    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:30:33.772982 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.772954    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:30:33.971446 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:33.971409    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:34.033603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:34.033581    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"]
Sep 08 14:30:34.035982 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:30:34.035956    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb66ead7b_a5c7_48c7_87ed_502eb4507230.slice/crio-40efb743531f692a3d9498cfb0423097fbf6f3cc8ceaac420ceb1a8237af0fc3 WatchSource:0}: Error finding container 40efb743531f692a3d9498cfb0423097fbf6f3cc8ceaac420ceb1a8237af0fc3: Status 404 returned error can't find the container with id 40efb743531f692a3d9498cfb0423097fbf6f3cc8ceaac420ceb1a8237af0fc3
Sep 08 14:30:34.059125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:34.059071    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"]
Sep 08 14:30:34.079353 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:30:34.079330    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd458ea6_161b_489e_b07d_905be3ef51eb.slice/crio-00e5ee9b186ef23ecccebc8dbe1a6c2943967051ff99d25bf49d5d9a9082a8a4 WatchSource:0}: Error finding container 00e5ee9b186ef23ecccebc8dbe1a6c2943967051ff99d25bf49d5d9a9082a8a4: Status 404 returned error can't find the container with id 00e5ee9b186ef23ecccebc8dbe1a6c2943967051ff99d25bf49d5d9a9082a8a4
Sep 08 14:30:34.792729 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:34.792700    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:34.924122 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:34.924090    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:35.016589 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:35.016555    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz" event={"ID":"fd458ea6-161b-489e-b07d-905be3ef51eb","Type":"ContainerStarted","Data":"7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035"}
Sep 08 14:30:35.016787 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:35.016613    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz" event={"ID":"fd458ea6-161b-489e-b07d-905be3ef51eb","Type":"ContainerStarted","Data":"00e5ee9b186ef23ecccebc8dbe1a6c2943967051ff99d25bf49d5d9a9082a8a4"}
Sep 08 14:30:35.016787 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:35.016641    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:30:35.018303 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:35.018272    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s" event={"ID":"b66ead7b-a5c7-48c7-87ed-502eb4507230","Type":"ContainerStarted","Data":"ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8"}
Sep 08 14:30:35.018417 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:35.018321    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s" event={"ID":"b66ead7b-a5c7-48c7-87ed-502eb4507230","Type":"ContainerStarted","Data":"40efb743531f692a3d9498cfb0423097fbf6f3cc8ceaac420ceb1a8237af0fc3"}
Sep 08 14:30:35.018417 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:35.018348    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:30:35.024033 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:35.024010    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:35.034730 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:35.034676    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz" podStartSLOduration=2.034661151 podStartE2EDuration="2.034661151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:30:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:30:35.033366266 +0000 UTC m=+850.264297466" watchObservedRunningTime="2026-09-08 14:30:35.034661151 +0000 UTC m=+850.265592347"
Sep 08 14:30:35.050865 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:35.050775    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s" podStartSLOduration=2.050759069 podStartE2EDuration="2.050759069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:30:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:30:35.048915104 +0000 UTC m=+850.279846301" watchObservedRunningTime="2026-09-08 14:30:35.050759069 +0000 UTC m=+850.281690266"
Sep 08 14:30:35.605448 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:35.605414    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:36.978810 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:36.978781    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:30:38.960531 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:38.960486    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:30:44.793911 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:44.793879    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:46.026325 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:46.026295    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:30:46.026877 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:46.026380    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:30:47.546792 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:47.546761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:51.176988 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:51.176954    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:51.803931 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:51.803893    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:51.827815 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:51.827788    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"]
Sep 08 14:30:51.828014 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:51.827992    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz" podUID="6e987115-a47a-46d4-ac46-938a75940b54" containerName="payload-processing" containerID="cri-o://578d30bc9e1b4e0320bc53871ab08bf89daf109d11b9a6507a0835ad82f34570" gracePeriod=30
Sep 08 14:30:51.833821 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:51.833795    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"]
Sep 08 14:30:51.834062 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:51.834019    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv" podUID="d523560a-6b69-4633-bcda-823a901a0c94" containerName="payload-pre-processing" containerID="cri-o://a2821fa146393a0b4f6721cb0474036aadc91736fd41a48290f261d5302b8e68" gracePeriod=30
Sep 08 14:30:51.898860 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:51.898832    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:52.000291 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.000184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:52.099614 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.099584    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:52.103886 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.103862    2558 generic.go:356] "Generic (PLEG): container finished" podID="6e987115-a47a-46d4-ac46-938a75940b54" containerID="578d30bc9e1b4e0320bc53871ab08bf89daf109d11b9a6507a0835ad82f34570" exitCode=0
Sep 08 14:30:52.104010 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.103931    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz" event={"ID":"6e987115-a47a-46d4-ac46-938a75940b54","Type":"ContainerDied","Data":"578d30bc9e1b4e0320bc53871ab08bf89daf109d11b9a6507a0835ad82f34570"}
Sep 08 14:30:52.104010 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.103982    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz" event={"ID":"6e987115-a47a-46d4-ac46-938a75940b54","Type":"ContainerDied","Data":"3c782c0bd356e199b15c8a225adefa31df33904186e2c9ed835bc1e892dadec2"}
Sep 08 14:30:52.104010 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.104000    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c782c0bd356e199b15c8a225adefa31df33904186e2c9ed835bc1e892dadec2"
Sep 08 14:30:52.105338 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.105318    2558 generic.go:356] "Generic (PLEG): container finished" podID="d523560a-6b69-4633-bcda-823a901a0c94" containerID="a2821fa146393a0b4f6721cb0474036aadc91736fd41a48290f261d5302b8e68" exitCode=0
Sep 08 14:30:52.105436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.105360    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv" event={"ID":"d523560a-6b69-4633-bcda-823a901a0c94","Type":"ContainerDied","Data":"a2821fa146393a0b4f6721cb0474036aadc91736fd41a48290f261d5302b8e68"}
Sep 08 14:30:52.105436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.105402    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv" event={"ID":"d523560a-6b69-4633-bcda-823a901a0c94","Type":"ContainerDied","Data":"cae0dbe94581c86e649eac7d872e30ea76774dba968d8cf453fe53d856090da9"}
Sep 08 14:30:52.105436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.105425    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cae0dbe94581c86e649eac7d872e30ea76774dba968d8cf453fe53d856090da9"
Sep 08 14:30:52.108681 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.108660    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:52.114672 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.114655    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:52.197971 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.197937    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:52.267979 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.267944    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d523560a-6b69-4633-bcda-823a901a0c94-plugins-config-volume\") pod \"d523560a-6b69-4633-bcda-823a901a0c94\" (UID: \"d523560a-6b69-4633-bcda-823a901a0c94\") "
Sep 08 14:30:52.268119 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.268000    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bc8m\" (UniqueName: \"kubernetes.io/projected/6e987115-a47a-46d4-ac46-938a75940b54-kube-api-access-7bc8m\") pod \"6e987115-a47a-46d4-ac46-938a75940b54\" (UID: \"6e987115-a47a-46d4-ac46-938a75940b54\") "
Sep 08 14:30:52.268119 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.268098    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpd5v\" (UniqueName: \"kubernetes.io/projected/d523560a-6b69-4633-bcda-823a901a0c94-kube-api-access-wpd5v\") pod \"d523560a-6b69-4633-bcda-823a901a0c94\" (UID: \"d523560a-6b69-4633-bcda-823a901a0c94\") "
Sep 08 14:30:52.268241 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.268128    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e987115-a47a-46d4-ac46-938a75940b54-plugins-config-volume\") pod \"6e987115-a47a-46d4-ac46-938a75940b54\" (UID: \"6e987115-a47a-46d4-ac46-938a75940b54\") "
Sep 08 14:30:52.268494 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.268464    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d523560a-6b69-4633-bcda-823a901a0c94-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d523560a-6b69-4633-bcda-823a901a0c94" (UID: "d523560a-6b69-4633-bcda-823a901a0c94"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:30:52.268608 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.268538    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e987115-a47a-46d4-ac46-938a75940b54-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6e987115-a47a-46d4-ac46-938a75940b54" (UID: "6e987115-a47a-46d4-ac46-938a75940b54"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:30:52.270258 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.270237    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d523560a-6b69-4633-bcda-823a901a0c94-kube-api-access-wpd5v" (OuterVolumeSpecName: "kube-api-access-wpd5v") pod "d523560a-6b69-4633-bcda-823a901a0c94" (UID: "d523560a-6b69-4633-bcda-823a901a0c94"). InnerVolumeSpecName "kube-api-access-wpd5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:30:52.270344 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.270303    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e987115-a47a-46d4-ac46-938a75940b54-kube-api-access-7bc8m" (OuterVolumeSpecName: "kube-api-access-7bc8m") pod "6e987115-a47a-46d4-ac46-938a75940b54" (UID: "6e987115-a47a-46d4-ac46-938a75940b54"). InnerVolumeSpecName "kube-api-access-7bc8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:30:52.369721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.369642    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d523560a-6b69-4633-bcda-823a901a0c94-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:30:52.369721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.369670    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bc8m\" (UniqueName: \"kubernetes.io/projected/6e987115-a47a-46d4-ac46-938a75940b54-kube-api-access-7bc8m\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:30:52.369721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.369680    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpd5v\" (UniqueName: \"kubernetes.io/projected/d523560a-6b69-4633-bcda-823a901a0c94-kube-api-access-wpd5v\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:30:52.369721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.369689    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e987115-a47a-46d4-ac46-938a75940b54-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:30:52.661447 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.661357    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:52.762715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.762679    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:52.878793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:52.878758    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:53.119822 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:53.119788    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"
Sep 08 14:30:53.119981 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:53.119792    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"
Sep 08 14:30:53.147526 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:53.147497    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"]
Sep 08 14:30:53.153751 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:53.153728    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32df8e95-5c59d49c94-cxnpv"]
Sep 08 14:30:53.171002 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:53.170980    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"]
Sep 08 14:30:53.183506 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:53.183486    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32df8e95-6645d5498c-s2clz"]
Sep 08 14:30:53.215884 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:53.215854    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:53.288451 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:53.288422    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e987115-a47a-46d4-ac46-938a75940b54" path="/var/lib/kubelet/pods/6e987115-a47a-46d4-ac46-938a75940b54/volumes"
Sep 08 14:30:53.288792 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:53.288780    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d523560a-6b69-4633-bcda-823a901a0c94" path="/var/lib/kubelet/pods/d523560a-6b69-4633-bcda-823a901a0c94/volumes"
Sep 08 14:30:53.787035 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:53.787001    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:53.973581 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:53.973546    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:56.375761 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:56.375724    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:56.475852 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:56.475822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:56.579130 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:56.579099    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:56.680057 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:56.679980    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:56.741308 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:56.741278    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:57.102142 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.102110    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:57.180665 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.180623    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:30:57.208305 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.208275    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"]
Sep 08 14:30:57.208784 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.208767    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d523560a-6b69-4633-bcda-823a901a0c94" containerName="payload-pre-processing"
Sep 08 14:30:57.208784 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.208783    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d523560a-6b69-4633-bcda-823a901a0c94" containerName="payload-pre-processing"
Sep 08 14:30:57.208910 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.208803    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e987115-a47a-46d4-ac46-938a75940b54" containerName="payload-processing"
Sep 08 14:30:57.208910 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.208809    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e987115-a47a-46d4-ac46-938a75940b54" containerName="payload-processing"
Sep 08 14:30:57.208910 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.208865    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e987115-a47a-46d4-ac46-938a75940b54" containerName="payload-processing"
Sep 08 14:30:57.208910 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.208875    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d523560a-6b69-4633-bcda-823a901a0c94" containerName="payload-pre-processing"
Sep 08 14:30:57.211393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.211375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:30:57.213797 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.213775    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-0416fe39\""
Sep 08 14:30:57.213914 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.213881    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-0416fe39-dockercfg-tpwhb\""
Sep 08 14:30:57.222522 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.222500    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"]
Sep 08 14:30:57.269581 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.269545    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"]
Sep 08 14:30:57.273763 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.273736    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:30:57.289300 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.289272    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"]
Sep 08 14:30:57.316945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.316919    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdzrn\" (UniqueName: \"kubernetes.io/projected/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-kube-api-access-pdzrn\") pod \"payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x\" (UID: \"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc\") " pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:30:57.317098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.316974    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x\" (UID: \"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc\") " pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:30:57.417991 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.417916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2rgf\" (UniqueName: \"kubernetes.io/projected/52a116ac-1b13-4a46-88d1-7d77580036f7-kube-api-access-l2rgf\") pod \"payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5\" (UID: \"52a116ac-1b13-4a46-88d1-7d77580036f7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:30:57.417991 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.417971    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdzrn\" (UniqueName: \"kubernetes.io/projected/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-kube-api-access-pdzrn\") pod \"payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x\" (UID: \"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc\") " pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:30:57.418506 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.418232    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x\" (UID: \"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc\") " pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:30:57.418506 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.418285    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52a116ac-1b13-4a46-88d1-7d77580036f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5\" (UID: \"52a116ac-1b13-4a46-88d1-7d77580036f7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:30:57.418866 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.418845    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x\" (UID: \"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc\") " pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:30:57.431304 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.431281    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdzrn\" (UniqueName: \"kubernetes.io/projected/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-kube-api-access-pdzrn\") pod \"payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x\" (UID: \"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc\") " pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:30:57.518754 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.518713    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52a116ac-1b13-4a46-88d1-7d77580036f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5\" (UID: \"52a116ac-1b13-4a46-88d1-7d77580036f7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:30:57.518923 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.518812    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2rgf\" (UniqueName: \"kubernetes.io/projected/52a116ac-1b13-4a46-88d1-7d77580036f7-kube-api-access-l2rgf\") pod \"payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5\" (UID: \"52a116ac-1b13-4a46-88d1-7d77580036f7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:30:57.519568 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.519547    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52a116ac-1b13-4a46-88d1-7d77580036f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5\" (UID: \"52a116ac-1b13-4a46-88d1-7d77580036f7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:30:57.522257 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.522238    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:30:57.527694 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.527668    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2rgf\" (UniqueName: \"kubernetes.io/projected/52a116ac-1b13-4a46-88d1-7d77580036f7-kube-api-access-l2rgf\") pod \"payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5\" (UID: \"52a116ac-1b13-4a46-88d1-7d77580036f7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:30:57.587587 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.587518    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:30:57.793436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.793412    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"]
Sep 08 14:30:57.795135 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:30:57.795110    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod856bdb9b_44ab_4ee2_8fb1_d435c73ecccc.slice/crio-b72b87d09cd3a03a8d005705ba4f45ab6cb66b08ea19a824e982d0e01171059e WatchSource:0}: Error finding container b72b87d09cd3a03a8d005705ba4f45ab6cb66b08ea19a824e982d0e01171059e: Status 404 returned error can't find the container with id b72b87d09cd3a03a8d005705ba4f45ab6cb66b08ea19a824e982d0e01171059e
Sep 08 14:30:57.861429 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:57.861406    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"]
Sep 08 14:30:57.863736 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:30:57.863712    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a116ac_1b13_4a46_88d1_7d77580036f7.slice/crio-7d33f4d8d00b73736ebf6191fe96b014acd5bcccf1fc8c6b80a647e1b41ec054 WatchSource:0}: Error finding container 7d33f4d8d00b73736ebf6191fe96b014acd5bcccf1fc8c6b80a647e1b41ec054: Status 404 returned error can't find the container with id 7d33f4d8d00b73736ebf6191fe96b014acd5bcccf1fc8c6b80a647e1b41ec054
Sep 08 14:30:58.148934 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:58.148831    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5" event={"ID":"52a116ac-1b13-4a46-88d1-7d77580036f7","Type":"ContainerStarted","Data":"fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea"}
Sep 08 14:30:58.149088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:58.149028    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:30:58.149088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:58.149072    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5" event={"ID":"52a116ac-1b13-4a46-88d1-7d77580036f7","Type":"ContainerStarted","Data":"7d33f4d8d00b73736ebf6191fe96b014acd5bcccf1fc8c6b80a647e1b41ec054"}
Sep 08 14:30:58.150510 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:58.150480    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x" event={"ID":"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc","Type":"ContainerStarted","Data":"dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5"}
Sep 08 14:30:58.150607 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:58.150527    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x" event={"ID":"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc","Type":"ContainerStarted","Data":"b72b87d09cd3a03a8d005705ba4f45ab6cb66b08ea19a824e982d0e01171059e"}
Sep 08 14:30:58.150607 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:58.150560    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:30:58.167646 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:58.167598    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5" podStartSLOduration=1.167580551 podStartE2EDuration="1.167580551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:30:58.164919355 +0000 UTC m=+873.395850554" watchObservedRunningTime="2026-09-08 14:30:58.167580551 +0000 UTC m=+873.398511753"
Sep 08 14:30:58.182758 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:30:58.182718    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x" podStartSLOduration=1.1827063230000001 podStartE2EDuration="1.182706323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:30:58.180821381 +0000 UTC m=+873.411752580" watchObservedRunningTime="2026-09-08 14:30:58.182706323 +0000 UTC m=+873.413637512"
Sep 08 14:31:00.858363 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:00.858328    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:31:09.158262 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:09.158228    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:31:09.158675 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:09.158296    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:31:18.151003 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:18.150924    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:31:19.438149 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.438115    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"]
Sep 08 14:31:19.441503 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.441487    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.443852 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.443833    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-ef197403-kserve-self-signed-certs\""
Sep 08 14:31:19.449271 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.449250    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"]
Sep 08 14:31:19.513374 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.513341    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0fda38b-1fe9-490f-8438-5e74766132cf-tls-certs\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.513511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.513392    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkp44\" (UniqueName: \"kubernetes.io/projected/b0fda38b-1fe9-490f-8438-5e74766132cf-kube-api-access-lkp44\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.513511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.513444    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-home\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.513511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.513475    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-tmp-dir\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.513511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.513497    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-model-cache\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.513642 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.513529    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-dshm\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.614417 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.614383    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0fda38b-1fe9-490f-8438-5e74766132cf-tls-certs\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.614560 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.614433    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkp44\" (UniqueName: \"kubernetes.io/projected/b0fda38b-1fe9-490f-8438-5e74766132cf-kube-api-access-lkp44\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.614560 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.614471    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-home\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.614560 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.614506    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-tmp-dir\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.614560 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.614534    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-model-cache\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.614747 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.614585    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-dshm\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.614924 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.614903    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-tmp-dir\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.615000 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.614969    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-home\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.615054 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.615008    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-model-cache\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.616763 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.616730    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-dshm\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.616896 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.616877    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0fda38b-1fe9-490f-8438-5e74766132cf-tls-certs\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.622700 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.622682    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkp44\" (UniqueName: \"kubernetes.io/projected/b0fda38b-1fe9-490f-8438-5e74766132cf-kube-api-access-lkp44\") pod \"e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") " pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.751584 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.751556    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:19.847989 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:19.847962    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:20.019336 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:20.019311    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"]
Sep 08 14:31:20.021713 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:31:20.021670    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0fda38b_1fe9_490f_8438_5e74766132cf.slice/crio-3020e7f8901daf2a386c841170dd79ec9b31cb158ef11d0a5a17fc59cc3a5886 WatchSource:0}: Error finding container 3020e7f8901daf2a386c841170dd79ec9b31cb158ef11d0a5a17fc59cc3a5886: Status 404 returned error can't find the container with id 3020e7f8901daf2a386c841170dd79ec9b31cb158ef11d0a5a17fc59cc3a5886
Sep 08 14:31:20.265706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:20.265671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn" event={"ID":"b0fda38b-1fe9-490f-8438-5e74766132cf","Type":"ContainerStarted","Data":"dc87db44fb15d95e8349ffffdf34ae38424a28e7e7b989d10a426d39f3d6ef79"}
Sep 08 14:31:20.265867 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:20.265713    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn" event={"ID":"b0fda38b-1fe9-490f-8438-5e74766132cf","Type":"ContainerStarted","Data":"3020e7f8901daf2a386c841170dd79ec9b31cb158ef11d0a5a17fc59cc3a5886"}
Sep 08 14:31:20.283352 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:20.283253    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn" podStartSLOduration=1.2832396959999999 podStartE2EDuration="1.283239696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:31:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:31:20.281609908 +0000 UTC m=+895.512541102" watchObservedRunningTime="2026-09-08 14:31:20.283239696 +0000 UTC m=+895.514170892"
Sep 08 14:31:27.899872 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:27.899838    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:27.938160 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:27.938098    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"]
Sep 08 14:31:27.938421 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:27.938370    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x" podUID="856bdb9b-44ab-4ee2-8fb1-d435c73ecccc" containerName="payload-processing" containerID="cri-o://dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5" gracePeriod=30
Sep 08 14:31:27.944542 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:27.944504    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"]
Sep 08 14:31:27.945495 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:27.945265    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5" podUID="52a116ac-1b13-4a46-88d1-7d77580036f7" containerName="payload-pre-processing" containerID="cri-o://fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea" gracePeriod=30
Sep 08 14:31:27.978378 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:27.978337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:28.038430 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.038398    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:28.123808 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.123775    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:28.187246 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.187192    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:28.218730 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.218705    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:31:28.221607 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.221588    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:31:28.300580 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.300549    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2rgf\" (UniqueName: \"kubernetes.io/projected/52a116ac-1b13-4a46-88d1-7d77580036f7-kube-api-access-l2rgf\") pod \"52a116ac-1b13-4a46-88d1-7d77580036f7\" (UID: \"52a116ac-1b13-4a46-88d1-7d77580036f7\") "
Sep 08 14:31:28.300730 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.300609    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-plugins-config-volume\") pod \"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc\" (UID: \"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc\") "
Sep 08 14:31:28.300730 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.300647    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdzrn\" (UniqueName: \"kubernetes.io/projected/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-kube-api-access-pdzrn\") pod \"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc\" (UID: \"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc\") "
Sep 08 14:31:28.300818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.300792    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52a116ac-1b13-4a46-88d1-7d77580036f7-plugins-config-volume\") pod \"52a116ac-1b13-4a46-88d1-7d77580036f7\" (UID: \"52a116ac-1b13-4a46-88d1-7d77580036f7\") "
Sep 08 14:31:28.301020 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.300992    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "856bdb9b-44ab-4ee2-8fb1-d435c73ecccc" (UID: "856bdb9b-44ab-4ee2-8fb1-d435c73ecccc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:31:28.301172 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.301139    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:31:28.301270 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.301196    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52a116ac-1b13-4a46-88d1-7d77580036f7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "52a116ac-1b13-4a46-88d1-7d77580036f7" (UID: "52a116ac-1b13-4a46-88d1-7d77580036f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:31:28.302786 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.302754    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-kube-api-access-pdzrn" (OuterVolumeSpecName: "kube-api-access-pdzrn") pod "856bdb9b-44ab-4ee2-8fb1-d435c73ecccc" (UID: "856bdb9b-44ab-4ee2-8fb1-d435c73ecccc"). InnerVolumeSpecName "kube-api-access-pdzrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:31:28.302899 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.302823    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a116ac-1b13-4a46-88d1-7d77580036f7-kube-api-access-l2rgf" (OuterVolumeSpecName: "kube-api-access-l2rgf") pod "52a116ac-1b13-4a46-88d1-7d77580036f7" (UID: "52a116ac-1b13-4a46-88d1-7d77580036f7"). InnerVolumeSpecName "kube-api-access-l2rgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:31:28.303570 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.303548    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:28.306693 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.306671    2558 generic.go:356] "Generic (PLEG): container finished" podID="52a116ac-1b13-4a46-88d1-7d77580036f7" containerID="fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea" exitCode=0
Sep 08 14:31:28.306772 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.306707    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"
Sep 08 14:31:28.306772 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.306746    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5" event={"ID":"52a116ac-1b13-4a46-88d1-7d77580036f7","Type":"ContainerDied","Data":"fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea"}
Sep 08 14:31:28.306856 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.306788    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5" event={"ID":"52a116ac-1b13-4a46-88d1-7d77580036f7","Type":"ContainerDied","Data":"7d33f4d8d00b73736ebf6191fe96b014acd5bcccf1fc8c6b80a647e1b41ec054"}
Sep 08 14:31:28.306856 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.306816    2558 scope.go:117] "RemoveContainer" containerID="fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea"
Sep 08 14:31:28.308195 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.308174    2558 generic.go:356] "Generic (PLEG): container finished" podID="856bdb9b-44ab-4ee2-8fb1-d435c73ecccc" containerID="dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5" exitCode=0
Sep 08 14:31:28.308313 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.308232    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"
Sep 08 14:31:28.308313 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.308254    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x" event={"ID":"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc","Type":"ContainerDied","Data":"dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5"}
Sep 08 14:31:28.308313 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.308301    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x" event={"ID":"856bdb9b-44ab-4ee2-8fb1-d435c73ecccc","Type":"ContainerDied","Data":"b72b87d09cd3a03a8d005705ba4f45ab6cb66b08ea19a824e982d0e01171059e"}
Sep 08 14:31:28.315713 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.315695    2558 scope.go:117] "RemoveContainer" containerID="fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea"
Sep 08 14:31:28.316005 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:31:28.315982    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea\": container with ID starting with fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea not found: ID does not exist" containerID="fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea"
Sep 08 14:31:28.316062 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.316018    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea"} err="failed to get container status \"fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea\": rpc error: code = NotFound desc = could not find container \"fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea\": container with ID starting with fffc3dd9039c45b066704bd5c01040678150ae284633998f4544bc73771b09ea not found: ID does not exist"
Sep 08 14:31:28.316062 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.316043    2558 scope.go:117] "RemoveContainer" containerID="dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5"
Sep 08 14:31:28.323809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.323794    2558 scope.go:117] "RemoveContainer" containerID="dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5"
Sep 08 14:31:28.324022 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:31:28.324007    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5\": container with ID starting with dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5 not found: ID does not exist" containerID="dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5"
Sep 08 14:31:28.324071 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.324028    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5"} err="failed to get container status \"dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5\": rpc error: code = NotFound desc = could not find container \"dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5\": container with ID starting with dd9f6d169087cb527006d4e262ee225e753fc7e483171290660ce4c2a89becd5 not found: ID does not exist"
Sep 08 14:31:28.333343 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.333322    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"]
Sep 08 14:31:28.338156 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.338131    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0416fe39-6b5586f6f8-xnf9x"]
Sep 08 14:31:28.350283 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.350265    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"]
Sep 08 14:31:28.354503 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.354483    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0416fe39-59db87bdfd-wv9d5"]
Sep 08 14:31:28.402676 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.402620    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52a116ac-1b13-4a46-88d1-7d77580036f7-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:31:28.402676 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.402641    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2rgf\" (UniqueName: \"kubernetes.io/projected/52a116ac-1b13-4a46-88d1-7d77580036f7-kube-api-access-l2rgf\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:31:28.402676 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.402651    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdzrn\" (UniqueName: \"kubernetes.io/projected/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc-kube-api-access-pdzrn\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:31:28.708501 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:28.708414    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:29.092182 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:29.092143    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:29.289898 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:29.289855    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52a116ac-1b13-4a46-88d1-7d77580036f7" path="/var/lib/kubelet/pods/52a116ac-1b13-4a46-88d1-7d77580036f7/volumes"
Sep 08 14:31:29.290493 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:29.290471    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="856bdb9b-44ab-4ee2-8fb1-d435c73ecccc" path="/var/lib/kubelet/pods/856bdb9b-44ab-4ee2-8fb1-d435c73ecccc/volumes"
Sep 08 14:31:29.590309 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:29.590261    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:29.752196 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:29.752161    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:29.752367 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:29.752224    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:29.764721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:29.764692    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:30.329963 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:30.329933    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:31:31.376328 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:31.376292    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:31.476195 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:31.476155    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:32.686012 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:32.685983    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:32.788067 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:32.788030    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:32.847784 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:32.847755    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:33.189295 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.189264    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:33.289937 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.289903    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:33.320716 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.320683    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"]
Sep 08 14:31:33.321249 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.321232    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="856bdb9b-44ab-4ee2-8fb1-d435c73ecccc" containerName="payload-processing"
Sep 08 14:31:33.321310 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.321251    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="856bdb9b-44ab-4ee2-8fb1-d435c73ecccc" containerName="payload-processing"
Sep 08 14:31:33.321310 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.321296    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52a116ac-1b13-4a46-88d1-7d77580036f7" containerName="payload-pre-processing"
Sep 08 14:31:33.321310 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.321305    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="52a116ac-1b13-4a46-88d1-7d77580036f7" containerName="payload-pre-processing"
Sep 08 14:31:33.321414 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.321403    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="856bdb9b-44ab-4ee2-8fb1-d435c73ecccc" containerName="payload-processing"
Sep 08 14:31:33.321450 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.321422    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="52a116ac-1b13-4a46-88d1-7d77580036f7" containerName="payload-pre-processing"
Sep 08 14:31:33.324667 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.324649    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:31:33.327396 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.327373    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5298415e\""
Sep 08 14:31:33.327500 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.327382    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5298415e-dockercfg-g6mm7\""
Sep 08 14:31:33.333066 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.332981    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"]
Sep 08 14:31:33.373114 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.373084    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"]
Sep 08 14:31:33.376569 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.376553    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:31:33.386474 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.386441    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"]
Sep 08 14:31:33.450426 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.450341    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl\" (UID: \"d2c6d548-2377-4c35-9fbb-58c42a2cf86c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:31:33.450426 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.450396    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8swx\" (UniqueName: \"kubernetes.io/projected/3ecc7349-7334-477c-9098-c3cbc68a87ff-kube-api-access-j8swx\") pod \"payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52\" (UID: \"3ecc7349-7334-477c-9098-c3cbc68a87ff\") " pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:31:33.450625 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.450489    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ecc7349-7334-477c-9098-c3cbc68a87ff-plugins-config-volume\") pod \"payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52\" (UID: \"3ecc7349-7334-477c-9098-c3cbc68a87ff\") " pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:31:33.450625 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.450522    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8vn5\" (UniqueName: \"kubernetes.io/projected/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-kube-api-access-q8vn5\") pod \"payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl\" (UID: \"d2c6d548-2377-4c35-9fbb-58c42a2cf86c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:31:33.551517 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.551481    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ecc7349-7334-477c-9098-c3cbc68a87ff-plugins-config-volume\") pod \"payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52\" (UID: \"3ecc7349-7334-477c-9098-c3cbc68a87ff\") " pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:31:33.551693 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.551533    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8vn5\" (UniqueName: \"kubernetes.io/projected/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-kube-api-access-q8vn5\") pod \"payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl\" (UID: \"d2c6d548-2377-4c35-9fbb-58c42a2cf86c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:31:33.551693 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.551658    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl\" (UID: \"d2c6d548-2377-4c35-9fbb-58c42a2cf86c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:31:33.551818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.551789    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8swx\" (UniqueName: \"kubernetes.io/projected/3ecc7349-7334-477c-9098-c3cbc68a87ff-kube-api-access-j8swx\") pod \"payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52\" (UID: \"3ecc7349-7334-477c-9098-c3cbc68a87ff\") " pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:31:33.552442 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.552416    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl\" (UID: \"d2c6d548-2377-4c35-9fbb-58c42a2cf86c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:31:33.552556 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.552475    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ecc7349-7334-477c-9098-c3cbc68a87ff-plugins-config-volume\") pod \"payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52\" (UID: \"3ecc7349-7334-477c-9098-c3cbc68a87ff\") " pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:31:33.560425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.560393    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8vn5\" (UniqueName: \"kubernetes.io/projected/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-kube-api-access-q8vn5\") pod \"payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl\" (UID: \"d2c6d548-2377-4c35-9fbb-58c42a2cf86c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:31:33.560534 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.560516    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8swx\" (UniqueName: \"kubernetes.io/projected/3ecc7349-7334-477c-9098-c3cbc68a87ff-kube-api-access-j8swx\") pod \"payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52\" (UID: \"3ecc7349-7334-477c-9098-c3cbc68a87ff\") " pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:31:33.637276 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.637237    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:31:33.688223 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.688177    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:31:33.909046 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.909018    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"]
Sep 08 14:31:33.911006 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:31:33.910977    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ecc7349_7334_477c_9098_c3cbc68a87ff.slice/crio-c9577f578ad9c4d0855c827e589e76053186696b258853a18b443606b7f041b7 WatchSource:0}: Error finding container c9577f578ad9c4d0855c827e589e76053186696b258853a18b443606b7f041b7: Status 404 returned error can't find the container with id c9577f578ad9c4d0855c827e589e76053186696b258853a18b443606b7f041b7
Sep 08 14:31:33.965670 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:33.965645    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"]
Sep 08 14:31:33.967490 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:31:33.967466    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2c6d548_2377_4c35_9fbb_58c42a2cf86c.slice/crio-8878e45e6a004c35f0632c4fa3cfe0dbefb9ac29c4ffc911920eef597a97e49b WatchSource:0}: Error finding container 8878e45e6a004c35f0632c4fa3cfe0dbefb9ac29c4ffc911920eef597a97e49b: Status 404 returned error can't find the container with id 8878e45e6a004c35f0632c4fa3cfe0dbefb9ac29c4ffc911920eef597a97e49b
Sep 08 14:31:34.049839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:34.049808    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:31:34.341700 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:34.341664    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52" event={"ID":"3ecc7349-7334-477c-9098-c3cbc68a87ff","Type":"ContainerStarted","Data":"049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19"}
Sep 08 14:31:34.341904 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:34.341727    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52" event={"ID":"3ecc7349-7334-477c-9098-c3cbc68a87ff","Type":"ContainerStarted","Data":"c9577f578ad9c4d0855c827e589e76053186696b258853a18b443606b7f041b7"}
Sep 08 14:31:34.341904 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:34.341753    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:31:34.343240 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:34.343194    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl" event={"ID":"d2c6d548-2377-4c35-9fbb-58c42a2cf86c","Type":"ContainerStarted","Data":"1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82"}
Sep 08 14:31:34.343362 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:34.343257    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl" event={"ID":"d2c6d548-2377-4c35-9fbb-58c42a2cf86c","Type":"ContainerStarted","Data":"8878e45e6a004c35f0632c4fa3cfe0dbefb9ac29c4ffc911920eef597a97e49b"}
Sep 08 14:31:34.343362 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:34.343279    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:31:34.366408 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:34.366356    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52" podStartSLOduration=1.366343286 podStartE2EDuration="1.366343286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:31:34.365065943 +0000 UTC m=+909.595997140" watchObservedRunningTime="2026-09-08 14:31:34.366343286 +0000 UTC m=+909.597274483"
Sep 08 14:31:34.383827 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:34.383774    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl" podStartSLOduration=1.383760517 podStartE2EDuration="1.383760517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:31:34.382098412 +0000 UTC m=+909.613029611" watchObservedRunningTime="2026-09-08 14:31:34.383760517 +0000 UTC m=+909.614691713"
Sep 08 14:31:45.351182 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:45.351150    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:31:45.351662 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:45.351349    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:31:52.247071 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:52.247034    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:31:56.874094 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.874059    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"]
Sep 08 14:31:56.877571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.877552    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:56.881139 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.881115    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-47d144\"/\"autores-model-47d144-kserve-self-signed-certs\""
Sep 08 14:31:56.881303 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.881124    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-47d144\"/\"default-dockercfg-nct59\""
Sep 08 14:31:56.881303 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.881269    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-47d144\"/\"kube-root-ca.crt\""
Sep 08 14:31:56.881303 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.881128    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-47d144\"/\"openshift-service-ca.crt\""
Sep 08 14:31:56.887923 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.887899    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"]
Sep 08 14:31:56.899100 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.899069    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:56.978591 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.978554    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-model-cache\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:56.978793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.978605    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/525a9918-0791-477a-8a1b-3f298120c821-tls-certs\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:56.978793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.978660    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f86cn\" (UniqueName: \"kubernetes.io/projected/525a9918-0791-477a-8a1b-3f298120c821-kube-api-access-f86cn\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:56.978793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.978776    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-tmp-dir\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:56.978961 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.978826    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-dshm\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:56.978961 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:56.978869    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-home\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.079557 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.079518    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-dshm\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.079557 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.079558    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-home\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.079813 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.079614    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-model-cache\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.079813 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.079652    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/525a9918-0791-477a-8a1b-3f298120c821-tls-certs\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.079813 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.079673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f86cn\" (UniqueName: \"kubernetes.io/projected/525a9918-0791-477a-8a1b-3f298120c821-kube-api-access-f86cn\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.079813 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.079740    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-tmp-dir\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.080081 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.080054    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-home\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.080140 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.080086    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-model-cache\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.080140 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.080118    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-tmp-dir\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.081906 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.081882    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-dshm\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.082318 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.082294    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/525a9918-0791-477a-8a1b-3f298120c821-tls-certs\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.088109 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.088086    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f86cn\" (UniqueName: \"kubernetes.io/projected/525a9918-0791-477a-8a1b-3f298120c821-kube-api-access-f86cn\") pod \"autores-model-47d144-kserve-7d99dc9cf-n92zd\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") " pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.189592 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.189519    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:31:57.464231 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:57.462743    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"]
Sep 08 14:31:57.470968 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:31:57.470936    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod525a9918_0791_477a_8a1b_3f298120c821.slice/crio-80ec021cce6ac3e0a4e24354d83f11dcb9d40e4c68d01b152e89078dd8b00136 WatchSource:0}: Error finding container 80ec021cce6ac3e0a4e24354d83f11dcb9d40e4c68d01b152e89078dd8b00136: Status 404 returned error can't find the container with id 80ec021cce6ac3e0a4e24354d83f11dcb9d40e4c68d01b152e89078dd8b00136
Sep 08 14:31:58.188043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:58.188004    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:31:58.470713 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:58.470606    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd" event={"ID":"525a9918-0791-477a-8a1b-3f298120c821","Type":"ContainerStarted","Data":"069beae8ecfcbc61da8503a0b1e44404ca45b2f092e4d4a269a3add2b4f14f78"}
Sep 08 14:31:58.470713 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:58.470653    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd" event={"ID":"525a9918-0791-477a-8a1b-3f298120c821","Type":"ContainerStarted","Data":"80ec021cce6ac3e0a4e24354d83f11dcb9d40e4c68d01b152e89078dd8b00136"}
Sep 08 14:31:58.487537 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:31:58.487480    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd" podStartSLOduration=2.487464781 podStartE2EDuration="2.487464781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:31:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:31:58.485845736 +0000 UTC m=+933.716776934" watchObservedRunningTime="2026-09-08 14:31:58.487464781 +0000 UTC m=+933.718395978"
Sep 08 14:32:03.078572 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.078538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:03.108370 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.108340    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"]
Sep 08 14:32:03.108590 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.108562    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52" podUID="3ecc7349-7334-477c-9098-c3cbc68a87ff" containerName="payload-processing" containerID="cri-o://049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19" gracePeriod=30
Sep 08 14:32:03.116457 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.116424    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"]
Sep 08 14:32:03.116630 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.116606    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl" podUID="d2c6d548-2377-4c35-9fbb-58c42a2cf86c" containerName="payload-pre-processing" containerID="cri-o://1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82" gracePeriod=30
Sep 08 14:32:03.150008 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.149982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:03.212309 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.212276    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:03.294821 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.293253    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:03.355506 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.355471    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:03.368401 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.368380    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:32:03.399218 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.399180    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:32:03.443397 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.443372    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-plugins-config-volume\") pod \"d2c6d548-2377-4c35-9fbb-58c42a2cf86c\" (UID: \"d2c6d548-2377-4c35-9fbb-58c42a2cf86c\") "
Sep 08 14:32:03.443574 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.443442    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8swx\" (UniqueName: \"kubernetes.io/projected/3ecc7349-7334-477c-9098-c3cbc68a87ff-kube-api-access-j8swx\") pod \"3ecc7349-7334-477c-9098-c3cbc68a87ff\" (UID: \"3ecc7349-7334-477c-9098-c3cbc68a87ff\") "
Sep 08 14:32:03.443574 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.443519    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8vn5\" (UniqueName: \"kubernetes.io/projected/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-kube-api-access-q8vn5\") pod \"d2c6d548-2377-4c35-9fbb-58c42a2cf86c\" (UID: \"d2c6d548-2377-4c35-9fbb-58c42a2cf86c\") "
Sep 08 14:32:03.443685 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.443580    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ecc7349-7334-477c-9098-c3cbc68a87ff-plugins-config-volume\") pod \"3ecc7349-7334-477c-9098-c3cbc68a87ff\" (UID: \"3ecc7349-7334-477c-9098-c3cbc68a87ff\") "
Sep 08 14:32:03.443795 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.443765    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d2c6d548-2377-4c35-9fbb-58c42a2cf86c" (UID: "d2c6d548-2377-4c35-9fbb-58c42a2cf86c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:32:03.443948 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.443927    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ecc7349-7334-477c-9098-c3cbc68a87ff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3ecc7349-7334-477c-9098-c3cbc68a87ff" (UID: "3ecc7349-7334-477c-9098-c3cbc68a87ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:32:03.444021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.443981    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:03.445391 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.445365    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ecc7349-7334-477c-9098-c3cbc68a87ff-kube-api-access-j8swx" (OuterVolumeSpecName: "kube-api-access-j8swx") pod "3ecc7349-7334-477c-9098-c3cbc68a87ff" (UID: "3ecc7349-7334-477c-9098-c3cbc68a87ff"). InnerVolumeSpecName "kube-api-access-j8swx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:32:03.445506 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.445488    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-kube-api-access-q8vn5" (OuterVolumeSpecName: "kube-api-access-q8vn5") pod "d2c6d548-2377-4c35-9fbb-58c42a2cf86c" (UID: "d2c6d548-2377-4c35-9fbb-58c42a2cf86c"). InnerVolumeSpecName "kube-api-access-q8vn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:32:03.497428 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.497399    2558 generic.go:356] "Generic (PLEG): container finished" podID="3ecc7349-7334-477c-9098-c3cbc68a87ff" containerID="049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19" exitCode=0
Sep 08 14:32:03.497571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.497446    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"
Sep 08 14:32:03.497571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.497482    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52" event={"ID":"3ecc7349-7334-477c-9098-c3cbc68a87ff","Type":"ContainerDied","Data":"049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19"}
Sep 08 14:32:03.497571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.497545    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52" event={"ID":"3ecc7349-7334-477c-9098-c3cbc68a87ff","Type":"ContainerDied","Data":"c9577f578ad9c4d0855c827e589e76053186696b258853a18b443606b7f041b7"}
Sep 08 14:32:03.497748 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.497577    2558 scope.go:117] "RemoveContainer" containerID="049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19"
Sep 08 14:32:03.498894 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.498867    2558 generic.go:356] "Generic (PLEG): container finished" podID="d2c6d548-2377-4c35-9fbb-58c42a2cf86c" containerID="1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82" exitCode=0
Sep 08 14:32:03.498982 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.498917    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"
Sep 08 14:32:03.498982 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.498950    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl" event={"ID":"d2c6d548-2377-4c35-9fbb-58c42a2cf86c","Type":"ContainerDied","Data":"1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82"}
Sep 08 14:32:03.499084 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.499000    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl" event={"ID":"d2c6d548-2377-4c35-9fbb-58c42a2cf86c","Type":"ContainerDied","Data":"8878e45e6a004c35f0632c4fa3cfe0dbefb9ac29c4ffc911920eef597a97e49b"}
Sep 08 14:32:03.506216 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.506190    2558 scope.go:117] "RemoveContainer" containerID="049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19"
Sep 08 14:32:03.506514 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:32:03.506492    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19\": container with ID starting with 049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19 not found: ID does not exist" containerID="049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19"
Sep 08 14:32:03.506597 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.506523    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19"} err="failed to get container status \"049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19\": rpc error: code = NotFound desc = could not find container \"049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19\": container with ID starting with 049971b1aaef3ea0b5aae9bd9a948eefd0b821ac210544cadeb69eda619fdd19 not found: ID does not exist"
Sep 08 14:32:03.506597 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.506545    2558 scope.go:117] "RemoveContainer" containerID="1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82"
Sep 08 14:32:03.514528 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.514513    2558 scope.go:117] "RemoveContainer" containerID="1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82"
Sep 08 14:32:03.514796 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:32:03.514777    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82\": container with ID starting with 1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82 not found: ID does not exist" containerID="1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82"
Sep 08 14:32:03.514864 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.514807    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82"} err="failed to get container status \"1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82\": rpc error: code = NotFound desc = could not find container \"1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82\": container with ID starting with 1bbc35782861e552e4228f77a14ca0e8cf7bf40d93720ab490ecd476b1ba0e82 not found: ID does not exist"
Sep 08 14:32:03.523976 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.523954    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"]
Sep 08 14:32:03.529183 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.529163    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5298415e-7d5b5b685c-57g52"]
Sep 08 14:32:03.541775 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.541751    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"]
Sep 08 14:32:03.543496 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.543477    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5298415e-5c4bcdd598-zvdtl"]
Sep 08 14:32:03.544847 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.544827    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8vn5\" (UniqueName: \"kubernetes.io/projected/d2c6d548-2377-4c35-9fbb-58c42a2cf86c-kube-api-access-q8vn5\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:03.544894 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.544850    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ecc7349-7334-477c-9098-c3cbc68a87ff-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:03.544894 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:03.544860    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8swx\" (UniqueName: \"kubernetes.io/projected/3ecc7349-7334-477c-9098-c3cbc68a87ff-kube-api-access-j8swx\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:04.429473 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:04.429434    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:04.886707 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:04.886660    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:05.204673 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:05.204597    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:05.288388 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:05.288356    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ecc7349-7334-477c-9098-c3cbc68a87ff" path="/var/lib/kubelet/pods/3ecc7349-7334-477c-9098-c3cbc68a87ff/volumes"
Sep 08 14:32:05.288796 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:05.288780    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2c6d548-2377-4c35-9fbb-58c42a2cf86c" path="/var/lib/kubelet/pods/d2c6d548-2377-4c35-9fbb-58c42a2cf86c/volumes"
Sep 08 14:32:05.845378 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:05.845346    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:07.190106 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:07.190073    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:32:07.190552 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:07.190126    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:32:07.202777 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:07.202751    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:32:07.532429 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:07.532400    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:32:07.627132 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:07.627099    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:07.717135 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:07.717102    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:09.015540 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.015504    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:09.116622 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.116583    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:09.282052 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.281960    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:09.477499 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.477464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:09.586878 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.586791    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"]
Sep 08 14:32:09.586878 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.586815    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:09.587340 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.587323    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2c6d548-2377-4c35-9fbb-58c42a2cf86c" containerName="payload-pre-processing"
Sep 08 14:32:09.587340 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.587342    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2c6d548-2377-4c35-9fbb-58c42a2cf86c" containerName="payload-pre-processing"
Sep 08 14:32:09.587489 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.587386    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ecc7349-7334-477c-9098-c3cbc68a87ff" containerName="payload-processing"
Sep 08 14:32:09.587489 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.587396    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ecc7349-7334-477c-9098-c3cbc68a87ff" containerName="payload-processing"
Sep 08 14:32:09.587589 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.587500    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2c6d548-2377-4c35-9fbb-58c42a2cf86c" containerName="payload-pre-processing"
Sep 08 14:32:09.587589 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.587518    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ecc7349-7334-477c-9098-c3cbc68a87ff" containerName="payload-processing"
Sep 08 14:32:09.592148 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.592128    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:09.595025 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.595006    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-c7c09416\""
Sep 08 14:32:09.595121 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.595006    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-c7c09416-dockercfg-c7btv\""
Sep 08 14:32:09.602846 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.602816    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"]
Sep 08 14:32:09.626323 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.626295    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"]
Sep 08 14:32:09.629729 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.629713    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:09.639039 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.639018    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"]
Sep 08 14:32:09.702278 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.702240    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvbm2\" (UniqueName: \"kubernetes.io/projected/7e481b14-283b-4865-9189-73e69579c7e9-kube-api-access-cvbm2\") pod \"payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb\" (UID: \"7e481b14-283b-4865-9189-73e69579c7e9\") " pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:09.702444 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.702287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvx9p\" (UniqueName: \"kubernetes.io/projected/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-kube-api-access-pvx9p\") pod \"payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq\" (UID: \"60ad4d2a-9afe-48e1-abfb-9f42b81cf530\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:09.702444 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.702417    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e481b14-283b-4865-9189-73e69579c7e9-plugins-config-volume\") pod \"payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb\" (UID: \"7e481b14-283b-4865-9189-73e69579c7e9\") " pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:09.702530 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.702510    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq\" (UID: \"60ad4d2a-9afe-48e1-abfb-9f42b81cf530\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:09.804065 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.804030    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvbm2\" (UniqueName: \"kubernetes.io/projected/7e481b14-283b-4865-9189-73e69579c7e9-kube-api-access-cvbm2\") pod \"payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb\" (UID: \"7e481b14-283b-4865-9189-73e69579c7e9\") " pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:09.804305 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.804083    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvx9p\" (UniqueName: \"kubernetes.io/projected/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-kube-api-access-pvx9p\") pod \"payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq\" (UID: \"60ad4d2a-9afe-48e1-abfb-9f42b81cf530\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:09.804305 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.804144    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e481b14-283b-4865-9189-73e69579c7e9-plugins-config-volume\") pod \"payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb\" (UID: \"7e481b14-283b-4865-9189-73e69579c7e9\") " pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:09.804305 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.804195    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq\" (UID: \"60ad4d2a-9afe-48e1-abfb-9f42b81cf530\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:09.805117 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.805091    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq\" (UID: \"60ad4d2a-9afe-48e1-abfb-9f42b81cf530\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:09.805158 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.805091    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e481b14-283b-4865-9189-73e69579c7e9-plugins-config-volume\") pod \"payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb\" (UID: \"7e481b14-283b-4865-9189-73e69579c7e9\") " pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:09.812870 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.812846    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvbm2\" (UniqueName: \"kubernetes.io/projected/7e481b14-283b-4865-9189-73e69579c7e9-kube-api-access-cvbm2\") pod \"payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb\" (UID: \"7e481b14-283b-4865-9189-73e69579c7e9\") " pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:09.813115 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.813092    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvx9p\" (UniqueName: \"kubernetes.io/projected/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-kube-api-access-pvx9p\") pod \"payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq\" (UID: \"60ad4d2a-9afe-48e1-abfb-9f42b81cf530\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:09.903927 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.903843    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:09.941468 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:09.941431    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:10.176344 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:10.176270    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"]
Sep 08 14:32:10.179304 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:32:10.179279    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ad4d2a_9afe_48e1_abfb_9f42b81cf530.slice/crio-45d02bae8dc30b8a961e4e54adebcdda1161be343ebf8347377c1f15c4c39c3e WatchSource:0}: Error finding container 45d02bae8dc30b8a961e4e54adebcdda1161be343ebf8347377c1f15c4c39c3e: Status 404 returned error can't find the container with id 45d02bae8dc30b8a961e4e54adebcdda1161be343ebf8347377c1f15c4c39c3e
Sep 08 14:32:10.221355 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:10.221331    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"]
Sep 08 14:32:10.235334 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:32:10.235300    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e481b14_283b_4865_9189_73e69579c7e9.slice/crio-dd98860b612c5548b3483fe696ec401fc418932f6f92ecd6f46a5d22c5557260 WatchSource:0}: Error finding container dd98860b612c5548b3483fe696ec401fc418932f6f92ecd6f46a5d22c5557260: Status 404 returned error can't find the container with id dd98860b612c5548b3483fe696ec401fc418932f6f92ecd6f46a5d22c5557260
Sep 08 14:32:10.542032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:10.541996    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq" event={"ID":"60ad4d2a-9afe-48e1-abfb-9f42b81cf530","Type":"ContainerStarted","Data":"4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1"}
Sep 08 14:32:10.542256 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:10.542062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq" event={"ID":"60ad4d2a-9afe-48e1-abfb-9f42b81cf530","Type":"ContainerStarted","Data":"45d02bae8dc30b8a961e4e54adebcdda1161be343ebf8347377c1f15c4c39c3e"}
Sep 08 14:32:10.542256 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:10.542097    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:10.543573 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:10.543553    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb" event={"ID":"7e481b14-283b-4865-9189-73e69579c7e9","Type":"ContainerStarted","Data":"131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1"}
Sep 08 14:32:10.543676 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:10.543587    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb" event={"ID":"7e481b14-283b-4865-9189-73e69579c7e9","Type":"ContainerStarted","Data":"dd98860b612c5548b3483fe696ec401fc418932f6f92ecd6f46a5d22c5557260"}
Sep 08 14:32:10.543676 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:10.543641    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:10.561827 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:10.561767    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq" podStartSLOduration=1.5617482969999998 podStartE2EDuration="1.561748297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:32:10.558239017 +0000 UTC m=+945.789170216" watchObservedRunningTime="2026-09-08 14:32:10.561748297 +0000 UTC m=+945.792679498"
Sep 08 14:32:10.574802 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:10.574760    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb" podStartSLOduration=1.57474592 podStartE2EDuration="1.57474592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:32:10.573694065 +0000 UTC m=+945.804625263" watchObservedRunningTime="2026-09-08 14:32:10.57474592 +0000 UTC m=+945.805677116"
Sep 08 14:32:13.160311 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:13.160269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:13.210617 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:13.210584    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:13.656506 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:13.656468    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:32:14.973410 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:14.973380    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:17.858519 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:17.858488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:18.435018 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:18.434986    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"]
Sep 08 14:32:18.461526 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:18.461485    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:21.551959 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:21.551929    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:21.552348 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:21.551999    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:22.047978 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:22.047943    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:24.464861 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:24.464830    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"]
Sep 08 14:32:24.482406 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:24.482373    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:24.860519 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:24.860483    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:31.074579 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:31.074545    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:33.455190 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.455130    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn" podUID="b0fda38b-1fe9-490f-8438-5e74766132cf" containerName="main" containerID="cri-o://dc87db44fb15d95e8349ffffdf34ae38424a28e7e7b989d10a426d39f3d6ef79" gracePeriod=45
Sep 08 14:32:33.668056 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.668023    2558 generic.go:356] "Generic (PLEG): container finished" podID="b0fda38b-1fe9-490f-8438-5e74766132cf" containerID="dc87db44fb15d95e8349ffffdf34ae38424a28e7e7b989d10a426d39f3d6ef79" exitCode=0
Sep 08 14:32:33.668201 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.668102    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn" event={"ID":"b0fda38b-1fe9-490f-8438-5e74766132cf","Type":"ContainerDied","Data":"dc87db44fb15d95e8349ffffdf34ae38424a28e7e7b989d10a426d39f3d6ef79"}
Sep 08 14:32:33.715164 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.715101    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:32:33.840475 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.840439    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-model-cache\") pod \"b0fda38b-1fe9-490f-8438-5e74766132cf\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") "
Sep 08 14:32:33.840645 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.840483    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkp44\" (UniqueName: \"kubernetes.io/projected/b0fda38b-1fe9-490f-8438-5e74766132cf-kube-api-access-lkp44\") pod \"b0fda38b-1fe9-490f-8438-5e74766132cf\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") "
Sep 08 14:32:33.840645 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.840513    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-tmp-dir\") pod \"b0fda38b-1fe9-490f-8438-5e74766132cf\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") "
Sep 08 14:32:33.840645 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.840539    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0fda38b-1fe9-490f-8438-5e74766132cf-tls-certs\") pod \"b0fda38b-1fe9-490f-8438-5e74766132cf\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") "
Sep 08 14:32:33.840645 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.840615    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-dshm\") pod \"b0fda38b-1fe9-490f-8438-5e74766132cf\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") "
Sep 08 14:32:33.840799 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.840672    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-home\") pod \"b0fda38b-1fe9-490f-8438-5e74766132cf\" (UID: \"b0fda38b-1fe9-490f-8438-5e74766132cf\") "
Sep 08 14:32:33.840852 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.840781    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0fda38b-1fe9-490f-8438-5e74766132cf" (UID: "b0fda38b-1fe9-490f-8438-5e74766132cf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:32:33.840911 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.840800    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b0fda38b-1fe9-490f-8438-5e74766132cf" (UID: "b0fda38b-1fe9-490f-8438-5e74766132cf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:32:33.841065 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.841039    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-home" (OuterVolumeSpecName: "home") pod "b0fda38b-1fe9-490f-8438-5e74766132cf" (UID: "b0fda38b-1fe9-490f-8438-5e74766132cf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:32:33.841156 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.841103    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-model-cache\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:33.841156 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.841117    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-tmp-dir\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:33.842882 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.842860    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0fda38b-1fe9-490f-8438-5e74766132cf-kube-api-access-lkp44" (OuterVolumeSpecName: "kube-api-access-lkp44") pod "b0fda38b-1fe9-490f-8438-5e74766132cf" (UID: "b0fda38b-1fe9-490f-8438-5e74766132cf"). InnerVolumeSpecName "kube-api-access-lkp44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:32:33.843130 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.843110    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-dshm" (OuterVolumeSpecName: "dshm") pod "b0fda38b-1fe9-490f-8438-5e74766132cf" (UID: "b0fda38b-1fe9-490f-8438-5e74766132cf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:32:33.843196 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.843118    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0fda38b-1fe9-490f-8438-5e74766132cf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b0fda38b-1fe9-490f-8438-5e74766132cf" (UID: "b0fda38b-1fe9-490f-8438-5e74766132cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:32:33.942542 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.942507    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkp44\" (UniqueName: \"kubernetes.io/projected/b0fda38b-1fe9-490f-8438-5e74766132cf-kube-api-access-lkp44\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:33.942542 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.942537    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0fda38b-1fe9-490f-8438-5e74766132cf-tls-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:33.942542 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.942549    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-dshm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:33.942778 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:33.942556    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0fda38b-1fe9-490f-8438-5e74766132cf-home\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:34.675067 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:34.675032    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"
Sep 08 14:32:34.675485 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:34.675036    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn" event={"ID":"b0fda38b-1fe9-490f-8438-5e74766132cf","Type":"ContainerDied","Data":"3020e7f8901daf2a386c841170dd79ec9b31cb158ef11d0a5a17fc59cc3a5886"}
Sep 08 14:32:34.675485 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:34.675160    2558 scope.go:117] "RemoveContainer" containerID="dc87db44fb15d95e8349ffffdf34ae38424a28e7e7b989d10a426d39f3d6ef79"
Sep 08 14:32:34.699510 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:34.699480    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"]
Sep 08 14:32:34.703697 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:34.703673    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-ef197403-kserve-76b6bb7846-n52jn"]
Sep 08 14:32:34.742748 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:34.742715    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:35.288446 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.288414    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0fda38b-1fe9-490f-8438-5e74766132cf" path="/var/lib/kubelet/pods/b0fda38b-1fe9-490f-8438-5e74766132cf/volumes"
Sep 08 14:32:35.345635 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.345602    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:35.384275 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.384238    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"]
Sep 08 14:32:35.385268 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.385230    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln" podUID="eafebe59-c6ca-4321-a6f2-8f8da114c2b7" containerName="payload-pre-processing" containerID="cri-o://49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340" gracePeriod=30
Sep 08 14:32:35.446865 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.446836    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:35.543902 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.543802    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:35.600684 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.600627    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:35.650633 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.650601    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:32:35.657026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.657006    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grpks\" (UniqueName: \"kubernetes.io/projected/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-kube-api-access-grpks\") pod \"eafebe59-c6ca-4321-a6f2-8f8da114c2b7\" (UID: \"eafebe59-c6ca-4321-a6f2-8f8da114c2b7\") "
Sep 08 14:32:35.657117 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.657043    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-plugins-config-volume\") pod \"eafebe59-c6ca-4321-a6f2-8f8da114c2b7\" (UID: \"eafebe59-c6ca-4321-a6f2-8f8da114c2b7\") "
Sep 08 14:32:35.657435 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.657410    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eafebe59-c6ca-4321-a6f2-8f8da114c2b7" (UID: "eafebe59-c6ca-4321-a6f2-8f8da114c2b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:32:35.659023 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.659002    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-kube-api-access-grpks" (OuterVolumeSpecName: "kube-api-access-grpks") pod "eafebe59-c6ca-4321-a6f2-8f8da114c2b7" (UID: "eafebe59-c6ca-4321-a6f2-8f8da114c2b7"). InnerVolumeSpecName "kube-api-access-grpks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:32:35.681430 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.681399    2558 generic.go:356] "Generic (PLEG): container finished" podID="eafebe59-c6ca-4321-a6f2-8f8da114c2b7" containerID="49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340" exitCode=0
Sep 08 14:32:35.681818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.681443    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"
Sep 08 14:32:35.681818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.681482    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln" event={"ID":"eafebe59-c6ca-4321-a6f2-8f8da114c2b7","Type":"ContainerDied","Data":"49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340"}
Sep 08 14:32:35.681818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.681561    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln" event={"ID":"eafebe59-c6ca-4321-a6f2-8f8da114c2b7","Type":"ContainerDied","Data":"d7836a95beb3a83bf0677d3bef1fce5feec90140e198a663baed6c55a46b2154"}
Sep 08 14:32:35.681818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.681589    2558 scope.go:117] "RemoveContainer" containerID="49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340"
Sep 08 14:32:35.691876 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.691856    2558 scope.go:117] "RemoveContainer" containerID="49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340"
Sep 08 14:32:35.692170 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:32:35.692151    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340\": container with ID starting with 49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340 not found: ID does not exist" containerID="49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340"
Sep 08 14:32:35.692231 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.692179    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340"} err="failed to get container status \"49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340\": rpc error: code = NotFound desc = could not find container \"49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340\": container with ID starting with 49f2ea04dac50f822f30687f193fe0ac13fddb9b4f364606887795e6ed86f340 not found: ID does not exist"
Sep 08 14:32:35.706850 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.706829    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"]
Sep 08 14:32:35.713349 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.713326    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-47d144-5fd95446cd-kkmln"]
Sep 08 14:32:35.757855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.757827    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grpks\" (UniqueName: \"kubernetes.io/projected/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-kube-api-access-grpks\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:35.757855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.757854    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eafebe59-c6ca-4321-a6f2-8f8da114c2b7-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:35.944759 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:35.944675    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:37.288628 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:37.288588    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eafebe59-c6ca-4321-a6f2-8f8da114c2b7" path="/var/lib/kubelet/pods/eafebe59-c6ca-4321-a6f2-8f8da114c2b7/volumes"
Sep 08 14:32:38.750075 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:38.750040    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:32:39.268847 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.268790    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:39.485011 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.484964    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd" podUID="525a9918-0791-477a-8a1b-3f298120c821" containerName="main" containerID="cri-o://069beae8ecfcbc61da8503a0b1e44404ca45b2f092e4d4a269a3add2b4f14f78" gracePeriod=45
Sep 08 14:32:39.705761 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.705727    2558 generic.go:356] "Generic (PLEG): container finished" podID="525a9918-0791-477a-8a1b-3f298120c821" containerID="069beae8ecfcbc61da8503a0b1e44404ca45b2f092e4d4a269a3add2b4f14f78" exitCode=0
Sep 08 14:32:39.705908 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.705802    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd" event={"ID":"525a9918-0791-477a-8a1b-3f298120c821","Type":"ContainerDied","Data":"069beae8ecfcbc61da8503a0b1e44404ca45b2f092e4d4a269a3add2b4f14f78"}
Sep 08 14:32:39.737931 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.737908    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:32:39.795336 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795307    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-tmp-dir\") pod \"525a9918-0791-477a-8a1b-3f298120c821\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") "
Sep 08 14:32:39.795753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795344    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-home\") pod \"525a9918-0791-477a-8a1b-3f298120c821\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") "
Sep 08 14:32:39.795753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795389    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-model-cache\") pod \"525a9918-0791-477a-8a1b-3f298120c821\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") "
Sep 08 14:32:39.795753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795406    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-dshm\") pod \"525a9918-0791-477a-8a1b-3f298120c821\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") "
Sep 08 14:32:39.795753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795456    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f86cn\" (UniqueName: \"kubernetes.io/projected/525a9918-0791-477a-8a1b-3f298120c821-kube-api-access-f86cn\") pod \"525a9918-0791-477a-8a1b-3f298120c821\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") "
Sep 08 14:32:39.795753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795524    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/525a9918-0791-477a-8a1b-3f298120c821-tls-certs\") pod \"525a9918-0791-477a-8a1b-3f298120c821\" (UID: \"525a9918-0791-477a-8a1b-3f298120c821\") "
Sep 08 14:32:39.795753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795565    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "525a9918-0791-477a-8a1b-3f298120c821" (UID: "525a9918-0791-477a-8a1b-3f298120c821"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:32:39.795753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795640    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-home" (OuterVolumeSpecName: "home") pod "525a9918-0791-477a-8a1b-3f298120c821" (UID: "525a9918-0791-477a-8a1b-3f298120c821"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:32:39.795753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795657    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-model-cache" (OuterVolumeSpecName: "model-cache") pod "525a9918-0791-477a-8a1b-3f298120c821" (UID: "525a9918-0791-477a-8a1b-3f298120c821"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:32:39.796087 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795848    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-tmp-dir\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:39.796087 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795868    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-home\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:39.796087 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.795881    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-model-cache\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:39.797680 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.797650    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/525a9918-0791-477a-8a1b-3f298120c821-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "525a9918-0791-477a-8a1b-3f298120c821" (UID: "525a9918-0791-477a-8a1b-3f298120c821"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:32:39.797775 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.797683    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-dshm" (OuterVolumeSpecName: "dshm") pod "525a9918-0791-477a-8a1b-3f298120c821" (UID: "525a9918-0791-477a-8a1b-3f298120c821"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:32:39.797775 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.797706    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/525a9918-0791-477a-8a1b-3f298120c821-kube-api-access-f86cn" (OuterVolumeSpecName: "kube-api-access-f86cn") pod "525a9918-0791-477a-8a1b-3f298120c821" (UID: "525a9918-0791-477a-8a1b-3f298120c821"). InnerVolumeSpecName "kube-api-access-f86cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:32:39.896409 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.896338    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/525a9918-0791-477a-8a1b-3f298120c821-tls-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:39.896409 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.896369    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/525a9918-0791-477a-8a1b-3f298120c821-dshm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:39.896409 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.896382    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f86cn\" (UniqueName: \"kubernetes.io/projected/525a9918-0791-477a-8a1b-3f298120c821-kube-api-access-f86cn\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:39.923751 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.923703    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:39.953086 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.953056    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"]
Sep 08 14:32:39.953372 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.953314    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb" podUID="7e481b14-283b-4865-9189-73e69579c7e9" containerName="payload-processing" containerID="cri-o://131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1" gracePeriod=30
Sep 08 14:32:39.959991 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.959959    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"]
Sep 08 14:32:39.960189 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.960162    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq" podUID="60ad4d2a-9afe-48e1-abfb-9f42b81cf530" containerName="payload-pre-processing" containerID="cri-o://4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1" gracePeriod=30
Sep 08 14:32:39.995075 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:39.995039    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:40.067754 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.067666    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:40.135537 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.135507    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:40.208638 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.208610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:40.220796 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.220774    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:40.230076 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.230055    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:40.278082 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.278052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:40.299412 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.299387    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvbm2\" (UniqueName: \"kubernetes.io/projected/7e481b14-283b-4865-9189-73e69579c7e9-kube-api-access-cvbm2\") pod \"7e481b14-283b-4865-9189-73e69579c7e9\" (UID: \"7e481b14-283b-4865-9189-73e69579c7e9\") "
Sep 08 14:32:40.299549 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.299453    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-plugins-config-volume\") pod \"60ad4d2a-9afe-48e1-abfb-9f42b81cf530\" (UID: \"60ad4d2a-9afe-48e1-abfb-9f42b81cf530\") "
Sep 08 14:32:40.299549 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.299490    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e481b14-283b-4865-9189-73e69579c7e9-plugins-config-volume\") pod \"7e481b14-283b-4865-9189-73e69579c7e9\" (UID: \"7e481b14-283b-4865-9189-73e69579c7e9\") "
Sep 08 14:32:40.299647 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.299554    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvx9p\" (UniqueName: \"kubernetes.io/projected/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-kube-api-access-pvx9p\") pod \"60ad4d2a-9afe-48e1-abfb-9f42b81cf530\" (UID: \"60ad4d2a-9afe-48e1-abfb-9f42b81cf530\") "
Sep 08 14:32:40.299848 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.299819    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "60ad4d2a-9afe-48e1-abfb-9f42b81cf530" (UID: "60ad4d2a-9afe-48e1-abfb-9f42b81cf530"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:32:40.299979 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.299887    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:40.299979 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.299882    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e481b14-283b-4865-9189-73e69579c7e9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7e481b14-283b-4865-9189-73e69579c7e9" (UID: "7e481b14-283b-4865-9189-73e69579c7e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:32:40.301596 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.301569    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e481b14-283b-4865-9189-73e69579c7e9-kube-api-access-cvbm2" (OuterVolumeSpecName: "kube-api-access-cvbm2") pod "7e481b14-283b-4865-9189-73e69579c7e9" (UID: "7e481b14-283b-4865-9189-73e69579c7e9"). InnerVolumeSpecName "kube-api-access-cvbm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:32:40.301748 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.301730    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-kube-api-access-pvx9p" (OuterVolumeSpecName: "kube-api-access-pvx9p") pod "60ad4d2a-9afe-48e1-abfb-9f42b81cf530" (UID: "60ad4d2a-9afe-48e1-abfb-9f42b81cf530"). InnerVolumeSpecName "kube-api-access-pvx9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:32:40.400568 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.400537    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e481b14-283b-4865-9189-73e69579c7e9-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:40.400568 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.400562    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvx9p\" (UniqueName: \"kubernetes.io/projected/60ad4d2a-9afe-48e1-abfb-9f42b81cf530-kube-api-access-pvx9p\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:40.400568 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.400572    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvbm2\" (UniqueName: \"kubernetes.io/projected/7e481b14-283b-4865-9189-73e69579c7e9-kube-api-access-cvbm2\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:32:40.718694 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.718657    2558 generic.go:356] "Generic (PLEG): container finished" podID="7e481b14-283b-4865-9189-73e69579c7e9" containerID="131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1" exitCode=0
Sep 08 14:32:40.718871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.718716    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"
Sep 08 14:32:40.718871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.718738    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb" event={"ID":"7e481b14-283b-4865-9189-73e69579c7e9","Type":"ContainerDied","Data":"131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1"}
Sep 08 14:32:40.718871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.718822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb" event={"ID":"7e481b14-283b-4865-9189-73e69579c7e9","Type":"ContainerDied","Data":"dd98860b612c5548b3483fe696ec401fc418932f6f92ecd6f46a5d22c5557260"}
Sep 08 14:32:40.718871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.718852    2558 scope.go:117] "RemoveContainer" containerID="131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1"
Sep 08 14:32:40.720248 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.720179    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"
Sep 08 14:32:40.720248 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.720192    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd" event={"ID":"525a9918-0791-477a-8a1b-3f298120c821","Type":"ContainerDied","Data":"80ec021cce6ac3e0a4e24354d83f11dcb9d40e4c68d01b152e89078dd8b00136"}
Sep 08 14:32:40.721736 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.721714    2558 generic.go:356] "Generic (PLEG): container finished" podID="60ad4d2a-9afe-48e1-abfb-9f42b81cf530" containerID="4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1" exitCode=0
Sep 08 14:32:40.721871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.721764    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"
Sep 08 14:32:40.721871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.721784    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq" event={"ID":"60ad4d2a-9afe-48e1-abfb-9f42b81cf530","Type":"ContainerDied","Data":"4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1"}
Sep 08 14:32:40.721871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.721820    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq" event={"ID":"60ad4d2a-9afe-48e1-abfb-9f42b81cf530","Type":"ContainerDied","Data":"45d02bae8dc30b8a961e4e54adebcdda1161be343ebf8347377c1f15c4c39c3e"}
Sep 08 14:32:40.728279 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.728259    2558 scope.go:117] "RemoveContainer" containerID="131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1"
Sep 08 14:32:40.728528 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:32:40.728508    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1\": container with ID starting with 131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1 not found: ID does not exist" containerID="131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1"
Sep 08 14:32:40.728594 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.728534    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1"} err="failed to get container status \"131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1\": rpc error: code = NotFound desc = could not find container \"131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1\": container with ID starting with 131a5337dc265451705dc6a3f8d86223dc409bc14cf325a08fec4542b06af2a1 not found: ID does not exist"
Sep 08 14:32:40.728594 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.728552    2558 scope.go:117] "RemoveContainer" containerID="069beae8ecfcbc61da8503a0b1e44404ca45b2f092e4d4a269a3add2b4f14f78"
Sep 08 14:32:40.737143 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.737128    2558 scope.go:117] "RemoveContainer" containerID="4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1"
Sep 08 14:32:40.745611 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.745593    2558 scope.go:117] "RemoveContainer" containerID="4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1"
Sep 08 14:32:40.745897 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:32:40.745874    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1\": container with ID starting with 4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1 not found: ID does not exist" containerID="4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1"
Sep 08 14:32:40.746024 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.745899    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1"} err="failed to get container status \"4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1\": rpc error: code = NotFound desc = could not find container \"4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1\": container with ID starting with 4a85c74ce564fcf15ba0fbc985589610ab130913ce23373527688285045056a1 not found: ID does not exist"
Sep 08 14:32:40.747215 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.747178    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"]
Sep 08 14:32:40.752370 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.752319    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c7c09416-7d66bdb5cc-244lb"]
Sep 08 14:32:40.762734 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.762712    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"]
Sep 08 14:32:40.766492 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.766472    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c7c09416-d799486c4-s4zzq"]
Sep 08 14:32:40.779825 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.779806    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"]
Sep 08 14:32:40.785174 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.785154    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-47d144/autores-model-47d144-kserve-7d99dc9cf-n92zd"]
Sep 08 14:32:40.989812 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:40.989776    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:41.090237 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:41.090130    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:41.189871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:41.189829    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:41.286925 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:41.286896    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="525a9918-0791-477a-8a1b-3f298120c821" path="/var/lib/kubelet/pods/525a9918-0791-477a-8a1b-3f298120c821/volumes"
Sep 08 14:32:41.287306 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:41.287288    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60ad4d2a-9afe-48e1-abfb-9f42b81cf530" path="/var/lib/kubelet/pods/60ad4d2a-9afe-48e1-abfb-9f42b81cf530/volumes"
Sep 08 14:32:41.287630 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:41.287616    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e481b14-283b-4865-9189-73e69579c7e9" path="/var/lib/kubelet/pods/7e481b14-283b-4865-9189-73e69579c7e9/volumes"
Sep 08 14:32:42.577508 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:42.577478    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:43.069494 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:43.069460    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:43.129538 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:43.129505    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:43.332904 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:43.332818    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:43.772007 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:43.771969    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:44.054433 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054360    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"]
Sep 08 14:32:44.054750 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054738    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ad4d2a-9afe-48e1-abfb-9f42b81cf530" containerName="payload-pre-processing"
Sep 08 14:32:44.054793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054753    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ad4d2a-9afe-48e1-abfb-9f42b81cf530" containerName="payload-pre-processing"
Sep 08 14:32:44.054793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054765    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eafebe59-c6ca-4321-a6f2-8f8da114c2b7" containerName="payload-pre-processing"
Sep 08 14:32:44.054793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054772    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="eafebe59-c6ca-4321-a6f2-8f8da114c2b7" containerName="payload-pre-processing"
Sep 08 14:32:44.054793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054785    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0fda38b-1fe9-490f-8438-5e74766132cf" containerName="main"
Sep 08 14:32:44.054793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054790    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0fda38b-1fe9-490f-8438-5e74766132cf" containerName="main"
Sep 08 14:32:44.054945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054802    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="525a9918-0791-477a-8a1b-3f298120c821" containerName="main"
Sep 08 14:32:44.054945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054808    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="525a9918-0791-477a-8a1b-3f298120c821" containerName="main"
Sep 08 14:32:44.054945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054820    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e481b14-283b-4865-9189-73e69579c7e9" containerName="payload-processing"
Sep 08 14:32:44.054945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054827    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e481b14-283b-4865-9189-73e69579c7e9" containerName="payload-processing"
Sep 08 14:32:44.054945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054883    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0fda38b-1fe9-490f-8438-5e74766132cf" containerName="main"
Sep 08 14:32:44.054945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054892    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e481b14-283b-4865-9189-73e69579c7e9" containerName="payload-processing"
Sep 08 14:32:44.054945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054899    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="525a9918-0791-477a-8a1b-3f298120c821" containerName="main"
Sep 08 14:32:44.054945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054906    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="60ad4d2a-9afe-48e1-abfb-9f42b81cf530" containerName="payload-pre-processing"
Sep 08 14:32:44.054945 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.054912    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="eafebe59-c6ca-4321-a6f2-8f8da114c2b7" containerName="payload-pre-processing"
Sep 08 14:32:44.059715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.059690    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.062116 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.062092    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-ae4649-openshift-default-dockercfg-wszq6\""
Sep 08 14:32:44.062265 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.062092    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 14:32:44.072405 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.072361    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"]
Sep 08 14:32:44.086535 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.086508    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:44.133987 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.133955    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-envoy\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.134124 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.133991    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-socket\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.134124 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.134010    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-credential-socket\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.134124 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.134043    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3c4f655-3238-4f74-bed2-906722d440a0-istiod-ca-cert\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.134124 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.134082    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5mbf\" (UniqueName: \"kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-kube-api-access-v5mbf\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.134124 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.134112    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-certs\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.134314 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.134172    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3c4f655-3238-4f74-bed2-906722d440a0-istio-podinfo\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.134314 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.134233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-data\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.134314 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.134256    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-istio-token\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.235473 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.235432    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-envoy\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.235473 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.235474    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-socket\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.235706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.235489    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-credential-socket\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.235706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.235512    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3c4f655-3238-4f74-bed2-906722d440a0-istiod-ca-cert\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.235706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.235559    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5mbf\" (UniqueName: \"kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-kube-api-access-v5mbf\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.235706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.235608    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-certs\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.235706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.235648    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3c4f655-3238-4f74-bed2-906722d440a0-istio-podinfo\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.235706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.235678    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-data\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.235706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.235707    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-istio-token\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.236089 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.235934    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-socket\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.236089 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.235969    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-credential-socket\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.236234 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.236134    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-data\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.236379 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.236354    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-certs\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.236442 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.236392    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3c4f655-3238-4f74-bed2-906722d440a0-istiod-ca-cert\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.237853 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.237829    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-envoy\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.238075 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.238053    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3c4f655-3238-4f74-bed2-906722d440a0-istio-podinfo\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.243900 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.243871    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-istio-token\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.244100 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.244082    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5mbf\" (UniqueName: \"kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-kube-api-access-v5mbf\") pod \"e2e-explicit-ae4649-openshift-default-546b687556-xqn96\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") " pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.375095 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.375010    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:44.559678 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.559642    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:44.649443 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.649420    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"]
Sep 08 14:32:44.651588 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:32:44.651561    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c4f655_3238_4f74_bed2_906722d440a0.slice/crio-5078150af87dc5d5d215f6635186fc3646cd2ce80e6b1f30af15dbe7d011a197 WatchSource:0}: Error finding container 5078150af87dc5d5d215f6635186fc3646cd2ce80e6b1f30af15dbe7d011a197: Status 404 returned error can't find the container with id 5078150af87dc5d5d215f6635186fc3646cd2ce80e6b1f30af15dbe7d011a197
Sep 08 14:32:44.653593 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.653573    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:32:44.747589 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:44.747554    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" event={"ID":"e3c4f655-3238-4f74-bed2-906722d440a0","Type":"ContainerStarted","Data":"5078150af87dc5d5d215f6635186fc3646cd2ce80e6b1f30af15dbe7d011a197"}
Sep 08 14:32:47.263470 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:47.263423    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:32:47.263837 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:47.263522    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:32:47.263837 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:47.263576    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:32:47.769651 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:47.769610    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" event={"ID":"e3c4f655-3238-4f74-bed2-906722d440a0","Type":"ContainerStarted","Data":"3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7"}
Sep 08 14:32:47.790984 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:47.790930    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" podStartSLOduration=1.18164233 podStartE2EDuration="3.790914943s" totalImagesPullingTime="2.609272613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:32:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:32:47.788580817 +0000 UTC m=+983.019512016" watchObservedRunningTime="2026-09-08 14:32:47.790914943 +0000 UTC m=+983.021846134"
Sep 08 14:32:48.375469 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:48.375429    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:48.380183 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:48.380155    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:48.775296 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:48.775250    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:48.776159 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:48.776134    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:32:49.531159 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:49.531125    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:51.294571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.294534    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:51.399929 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.399897    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:51.494951 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.494918    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:51.594991 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.594903    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:51.695627 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.695598    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:32:51.712830 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.712794    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"]
Sep 08 14:32:51.716813 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.716794    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:32:51.719694 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.719676    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-ae4649\""
Sep 08 14:32:51.719820 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.719699    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-ae4649-dockercfg-59n7r\""
Sep 08 14:32:51.729637 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.729616    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"]
Sep 08 14:32:51.810881 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.810850    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s\" (UID: \"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:32:51.811045 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.810903    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nwjf\" (UniqueName: \"kubernetes.io/projected/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-kube-api-access-8nwjf\") pod \"payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s\" (UID: \"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:32:51.912497 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.912414    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s\" (UID: \"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:32:51.912497 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.912475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nwjf\" (UniqueName: \"kubernetes.io/projected/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-kube-api-access-8nwjf\") pod \"payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s\" (UID: \"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:32:51.913131 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.913110    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s\" (UID: \"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:32:51.921357 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:51.921334    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nwjf\" (UniqueName: \"kubernetes.io/projected/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-kube-api-access-8nwjf\") pod \"payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s\" (UID: \"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:32:52.027160 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:52.027127    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:32:52.297498 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:52.297470    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"]
Sep 08 14:32:52.300034 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:32:52.300005    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5728f139_a6a7_49ff_8bf8_1a0ff7d6e9c4.slice/crio-e67cf8ee6d191934d6fa20b679dd0d553bc8bde2a93727c6e4f28c91f87977e5 WatchSource:0}: Error finding container e67cf8ee6d191934d6fa20b679dd0d553bc8bde2a93727c6e4f28c91f87977e5: Status 404 returned error can't find the container with id e67cf8ee6d191934d6fa20b679dd0d553bc8bde2a93727c6e4f28c91f87977e5
Sep 08 14:32:52.797604 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:52.797569    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s" event={"ID":"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4","Type":"ContainerStarted","Data":"a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4"}
Sep 08 14:32:52.797806 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:52.797624    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s" event={"ID":"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4","Type":"ContainerStarted","Data":"e67cf8ee6d191934d6fa20b679dd0d553bc8bde2a93727c6e4f28c91f87977e5"}
Sep 08 14:32:52.797806 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:52.797645    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:32:52.814174 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:52.814128    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s" podStartSLOduration=1.814114291 podStartE2EDuration="1.814114291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:32:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:32:52.813622323 +0000 UTC m=+988.044553521" watchObservedRunningTime="2026-09-08 14:32:52.814114291 +0000 UTC m=+988.045045488"
Sep 08 14:32:56.062345 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:32:56.062304    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:33:03.550825 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.550792    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"]
Sep 08 14:33:03.554198 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.554178    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.558627 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.558603    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2a2de3\"/\"default-dockercfg-jswvp\""
Sep 08 14:33:03.558768 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.558611    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2a2de3\"/\"test-model-2a2de3-kserve-self-signed-certs\""
Sep 08 14:33:03.558768 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.558610    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2a2de3\"/\"kube-root-ca.crt\""
Sep 08 14:33:03.558768 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.558649    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2a2de3\"/\"openshift-service-ca.crt\""
Sep 08 14:33:03.564581 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.564551    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"]
Sep 08 14:33:03.629044 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.629011    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-home\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.629250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.629078    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-model-cache\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.629250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.629100    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2knf\" (UniqueName: \"kubernetes.io/projected/44f13dae-ddb8-48fd-a480-1a813baf3b2d-kube-api-access-p2knf\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.629250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.629120    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-dshm\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.629250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.629142    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tmp-dir\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.629250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.629233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tls-certs\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.651884 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.651850    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:03.730329 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.730282    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-home\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.730521 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.730352    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-model-cache\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.730521 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.730368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2knf\" (UniqueName: \"kubernetes.io/projected/44f13dae-ddb8-48fd-a480-1a813baf3b2d-kube-api-access-p2knf\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.730521 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.730388    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-dshm\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.730521 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.730405    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tmp-dir\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.730521 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.730427    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tls-certs\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.730859 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.730826    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-home\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.730944 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.730831    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-model-cache\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.731003 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.730959    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tmp-dir\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.732881 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.732856    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-dshm\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.733428 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.733408    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tls-certs\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.739107 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.739079    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2knf\" (UniqueName: \"kubernetes.io/projected/44f13dae-ddb8-48fd-a480-1a813baf3b2d-kube-api-access-p2knf\") pod \"test-model-2a2de3-kserve-59656675bc-jc8lz\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") " pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:03.806043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.805953    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:33:03.864325 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:03.864289    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:04.138942 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:04.138917    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"]
Sep 08 14:33:04.141249 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:33:04.141185    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44f13dae_ddb8_48fd_a480_1a813baf3b2d.slice/crio-32b7ecffb648265e7d6f51bf1889441da312b4c6b304ec98189b5b21422cf5df WatchSource:0}: Error finding container 32b7ecffb648265e7d6f51bf1889441da312b4c6b304ec98189b5b21422cf5df: Status 404 returned error can't find the container with id 32b7ecffb648265e7d6f51bf1889441da312b4c6b304ec98189b5b21422cf5df
Sep 08 14:33:04.817371 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:04.817333    2558 patch_prober.go:28] interesting pod/e2e-explicit-ae4649-openshift-default-546b687556-xqn96 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 14:33:04.817743 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:04.817385    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 14:33:04.864700 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:04.864661    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz" event={"ID":"44f13dae-ddb8-48fd-a480-1a813baf3b2d","Type":"ContainerStarted","Data":"eb9cc82c8a80185ff2aadd835b54e516f38ba026b1bc42c7defbb615a6ea37f8"}
Sep 08 14:33:04.864865 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:04.864711    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz" event={"ID":"44f13dae-ddb8-48fd-a480-1a813baf3b2d","Type":"ContainerStarted","Data":"32b7ecffb648265e7d6f51bf1889441da312b4c6b304ec98189b5b21422cf5df"}
Sep 08 14:33:04.881121 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:04.881068    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz" podStartSLOduration=1.881044014 podStartE2EDuration="1.881044014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:33:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:33:04.880562698 +0000 UTC m=+1000.111493895" watchObservedRunningTime="2026-09-08 14:33:04.881044014 +0000 UTC m=+1000.111975211"
Sep 08 14:33:05.814567 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:05.814538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:05.871501 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:05.871469    2558 generic.go:356] "Generic (PLEG): container finished" podID="e3c4f655-3238-4f74-bed2-906722d440a0" containerID="3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7" exitCode=0
Sep 08 14:33:05.871895 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:05.871543    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" event={"ID":"e3c4f655-3238-4f74-bed2-906722d440a0","Type":"ContainerDied","Data":"3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7"}
Sep 08 14:33:05.872074 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:05.872054    2558 scope.go:117] "RemoveContainer" containerID="3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7"
Sep 08 14:33:05.873069 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:05.873043    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:33:05.873123 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:05.873110    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:33:05.873158 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:05.873140    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:33:06.879763 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:06.879725    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" event={"ID":"e3c4f655-3238-4f74-bed2-906722d440a0","Type":"ContainerStarted","Data":"d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8"}
Sep 08 14:33:07.375766 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:07.375725    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:33:07.375995 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:07.375968    2558 patch_prober.go:28] interesting pod/e2e-explicit-ae4649-openshift-default-546b687556-xqn96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 08 14:33:07.376048 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:07.376027    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 08 14:33:08.376298 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:08.376252    2558 patch_prober.go:28] interesting pod/e2e-explicit-ae4649-openshift-default-546b687556-xqn96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 08 14:33:08.376764 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:08.376318    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 08 14:33:09.375633 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:09.375588    2558 patch_prober.go:28] interesting pod/e2e-explicit-ae4649-openshift-default-546b687556-xqn96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 08 14:33:09.375821 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:09.375670    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 08 14:33:10.262529 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:10.262477    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:33:10.375697 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:10.375657    2558 patch_prober.go:28] interesting pod/e2e-explicit-ae4649-openshift-default-546b687556-xqn96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 08 14:33:10.375891 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:10.375717    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 08 14:33:11.375688 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:11.375648    2558 patch_prober.go:28] interesting pod/e2e-explicit-ae4649-openshift-default-546b687556-xqn96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 08 14:33:11.376065 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:11.375712    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 08 14:33:12.375957 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:12.375919    2558 patch_prober.go:28] interesting pod/e2e-explicit-ae4649-openshift-default-546b687556-xqn96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 08 14:33:12.376369 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:12.375977    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 08 14:33:13.379639 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:13.379607    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:33:13.380052 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:13.379892    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:33:13.380456 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:13.380435    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:33:13.864679 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:13.864635    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:13.864868 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:13.864691    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:13.877451 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:13.877425    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:13.928964 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:13.928936    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:33:14.751425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:14.751395    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:16.169680 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:16.169641    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:22.584315 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:22.584272    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:22.695610 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:22.695575    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:22.789469 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:22.789435    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:25.952140 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:25.952104    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:33:34.305224 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:34.305164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:37.662920 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:37.662887    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:38.690440 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:38.690397    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:41.057295 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:41.057257    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:33:44.795696 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:44.795665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:44.853989 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:44.853945    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:44.926495 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:44.926461    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:45.035370 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:45.035336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:52.966252 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:52.966200    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:55.507532 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:55.507494    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:33:56.258033 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:56.257998    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:33:56.318313 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:33:56.318279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:02.631425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:02.631375    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:02.737929 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:02.737895    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:02.834423 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:02.834388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:02.933926 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:02.933845    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:03.663465 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:03.663426    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:03.758035 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:03.758001    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:03.863935 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:03.863894    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"]
Sep 08 14:34:03.864188 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:03.864154    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s" podUID="5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4" containerName="payload-pre-processing" containerID="cri-o://a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4" gracePeriod=30
Sep 08 14:34:03.864464 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:03.864172    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:03.945736 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:03.945656    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:04.048497 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.048459    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:04.122745 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.122721    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:34:04.136320 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.136290    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:04.168679 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.168652    2558 generic.go:356] "Generic (PLEG): container finished" podID="5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4" containerID="a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4" exitCode=0
Sep 08 14:34:04.168818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.168703    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"
Sep 08 14:34:04.168818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.168740    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s" event={"ID":"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4","Type":"ContainerDied","Data":"a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4"}
Sep 08 14:34:04.168818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.168800    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s" event={"ID":"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4","Type":"ContainerDied","Data":"e67cf8ee6d191934d6fa20b679dd0d553bc8bde2a93727c6e4f28c91f87977e5"}
Sep 08 14:34:04.168928 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.168825    2558 scope.go:117] "RemoveContainer" containerID="a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4"
Sep 08 14:34:04.177325 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.177299    2558 scope.go:117] "RemoveContainer" containerID="a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4"
Sep 08 14:34:04.177585 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:04.177569    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4\": container with ID starting with a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4 not found: ID does not exist" containerID="a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4"
Sep 08 14:34:04.177635 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.177592    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4"} err="failed to get container status \"a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4\": rpc error: code = NotFound desc = could not find container \"a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4\": container with ID starting with a45e64364073e2094639e4e3eb18e3d16c95354a6d87a6b53b75b68e2ef1c2d4 not found: ID does not exist"
Sep 08 14:34:04.213409 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.213329    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-plugins-config-volume\") pod \"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4\" (UID: \"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4\") "
Sep 08 14:34:04.213409 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.213397    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nwjf\" (UniqueName: \"kubernetes.io/projected/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-kube-api-access-8nwjf\") pod \"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4\" (UID: \"5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4\") "
Sep 08 14:34:04.213709 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.213686    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4" (UID: "5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:34:04.215717 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.215682    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-kube-api-access-8nwjf" (OuterVolumeSpecName: "kube-api-access-8nwjf") pod "5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4" (UID: "5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4"). InnerVolumeSpecName "kube-api-access-8nwjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:34:04.314147 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.314108    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nwjf\" (UniqueName: \"kubernetes.io/projected/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-kube-api-access-8nwjf\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:34:04.314147 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.314147    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:34:04.412469 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.412429    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:04.491125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.491068    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"]
Sep 08 14:34:04.493399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:04.493372    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ae4649-5467cb86f4-ntz4s"]
Sep 08 14:34:05.287065 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:05.287026    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4" path="/var/lib/kubelet/pods/5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4/volumes"
Sep 08 14:34:05.345423 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:05.345390    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:09.149888 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:09.149845    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:34:10.961076 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:10.961033    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:11.085088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:11.085059    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:11.216306 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:11.216230    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"]
Sep 08 14:34:11.216460 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:11.216438    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy" containerID="cri-o://d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8" gracePeriod=30
Sep 08 14:34:11.243469 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:11.243435    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:11.677768 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:11.677738    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:11.925813 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:11.925779    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:12.017008 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:12.016971    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:13.258306 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:13.258269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:13.358872 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:13.358842    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:13.382247 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:13.382191    2558 patch_prober.go:28] interesting pod/e2e-explicit-ae4649-openshift-default-546b687556-xqn96 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 14:34:13.382398 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:13.382284    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 14:34:13.458815 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:13.458781    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:16.478938 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.478907    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:34:16.531422 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531392    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-socket\") pod \"e3c4f655-3238-4f74-bed2-906722d440a0\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") "
Sep 08 14:34:16.531607 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531436    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3c4f655-3238-4f74-bed2-906722d440a0-istiod-ca-cert\") pod \"e3c4f655-3238-4f74-bed2-906722d440a0\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") "
Sep 08 14:34:16.531607 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531454    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-certs\") pod \"e3c4f655-3238-4f74-bed2-906722d440a0\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") "
Sep 08 14:34:16.531607 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531477    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-istio-token\") pod \"e3c4f655-3238-4f74-bed2-906722d440a0\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") "
Sep 08 14:34:16.531607 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531540    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-credential-socket\") pod \"e3c4f655-3238-4f74-bed2-906722d440a0\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") "
Sep 08 14:34:16.531607 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531583    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5mbf\" (UniqueName: \"kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-kube-api-access-v5mbf\") pod \"e3c4f655-3238-4f74-bed2-906722d440a0\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") "
Sep 08 14:34:16.531872 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531658    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-data\") pod \"e3c4f655-3238-4f74-bed2-906722d440a0\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") "
Sep 08 14:34:16.531872 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531690    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-envoy\") pod \"e3c4f655-3238-4f74-bed2-906722d440a0\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") "
Sep 08 14:34:16.531872 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531727    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3c4f655-3238-4f74-bed2-906722d440a0-istio-podinfo\") pod \"e3c4f655-3238-4f74-bed2-906722d440a0\" (UID: \"e3c4f655-3238-4f74-bed2-906722d440a0\") "
Sep 08 14:34:16.531872 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531748    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e3c4f655-3238-4f74-bed2-906722d440a0" (UID: "e3c4f655-3238-4f74-bed2-906722d440a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:34:16.531872 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531780    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e3c4f655-3238-4f74-bed2-906722d440a0" (UID: "e3c4f655-3238-4f74-bed2-906722d440a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:34:16.532149 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531906    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "e3c4f655-3238-4f74-bed2-906722d440a0" (UID: "e3c4f655-3238-4f74-bed2-906722d440a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:34:16.532149 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.531918    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e3c4f655-3238-4f74-bed2-906722d440a0" (UID: "e3c4f655-3238-4f74-bed2-906722d440a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:34:16.532149 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.532013    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c4f655-3238-4f74-bed2-906722d440a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e3c4f655-3238-4f74-bed2-906722d440a0" (UID: "e3c4f655-3238-4f74-bed2-906722d440a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:34:16.532149 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.532107    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:34:16.532149 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.532121    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3c4f655-3238-4f74-bed2-906722d440a0-istiod-ca-cert\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:34:16.532149 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.532131    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-workload-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:34:16.532149 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.532139    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-credential-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:34:16.532149 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.532149    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-data\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:34:16.533913 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.533886    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3c4f655-3238-4f74-bed2-906722d440a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e3c4f655-3238-4f74-bed2-906722d440a0" (UID: "e3c4f655-3238-4f74-bed2-906722d440a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:34:16.534028 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.533889    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e3c4f655-3238-4f74-bed2-906722d440a0" (UID: "e3c4f655-3238-4f74-bed2-906722d440a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:34:16.534167 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.534148    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-kube-api-access-v5mbf" (OuterVolumeSpecName: "kube-api-access-v5mbf") pod "e3c4f655-3238-4f74-bed2-906722d440a0" (UID: "e3c4f655-3238-4f74-bed2-906722d440a0"). InnerVolumeSpecName "kube-api-access-v5mbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:34:16.534355 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.534335    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "e3c4f655-3238-4f74-bed2-906722d440a0" (UID: "e3c4f655-3238-4f74-bed2-906722d440a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:34:16.633226 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.633167    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3c4f655-3238-4f74-bed2-906722d440a0-istio-envoy\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:34:16.633226 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.633197    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3c4f655-3238-4f74-bed2-906722d440a0-istio-podinfo\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:34:16.633226 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.633227    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-istio-token\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:34:16.633226 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:16.633236    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5mbf\" (UniqueName: \"kubernetes.io/projected/e3c4f655-3238-4f74-bed2-906722d440a0-kube-api-access-v5mbf\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:34:17.243925 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.243887    2558 generic.go:356] "Generic (PLEG): container finished" podID="e3c4f655-3238-4f74-bed2-906722d440a0" containerID="d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8" exitCode=0
Sep 08 14:34:17.244154 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.244120    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" event={"ID":"e3c4f655-3238-4f74-bed2-906722d440a0","Type":"ContainerDied","Data":"d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8"}
Sep 08 14:34:17.244328 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.244177    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"
Sep 08 14:34:17.244328 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.244246    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96" event={"ID":"e3c4f655-3238-4f74-bed2-906722d440a0","Type":"ContainerDied","Data":"5078150af87dc5d5d215f6635186fc3646cd2ce80e6b1f30af15dbe7d011a197"}
Sep 08 14:34:17.244328 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.244253    2558 scope.go:117] "RemoveContainer" containerID="d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8"
Sep 08 14:34:17.253890 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.253871    2558 scope.go:117] "RemoveContainer" containerID="3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7"
Sep 08 14:34:17.264641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.264532    2558 scope.go:117] "RemoveContainer" containerID="d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8"
Sep 08 14:34:17.265195 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:17.265162    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8\": container with ID starting with d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8 not found: ID does not exist" containerID="d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8"
Sep 08 14:34:17.265327 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.265228    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8"} err="failed to get container status \"d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8\": rpc error: code = NotFound desc = could not find container \"d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8\": container with ID starting with d98fe022e1e9274d01e0b735978fa5dd6fd43168cd3ba13fcc43d3905cba1dd8 not found: ID does not exist"
Sep 08 14:34:17.265327 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.265254    2558 scope.go:117] "RemoveContainer" containerID="3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7"
Sep 08 14:34:17.265696 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:17.265667    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7\": container with ID starting with 3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7 not found: ID does not exist" containerID="3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7"
Sep 08 14:34:17.265803 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.265705    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7"} err="failed to get container status \"3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7\": rpc error: code = NotFound desc = could not find container \"3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7\": container with ID starting with 3ca105205cf5607ce9789f6206ec5424fa6783c173738e37ac2217c2cd9fa4e7 not found: ID does not exist"
Sep 08 14:34:17.267931 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.267912    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"]
Sep 08 14:34:17.272993 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.272968    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-ae4649-openshift-default-546b687556-xqn96"]
Sep 08 14:34:17.287534 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:17.287513    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" path="/var/lib/kubelet/pods/e3c4f655-3238-4f74-bed2-906722d440a0/volumes"
Sep 08 14:34:18.938016 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:18.937983    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:19.040870 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.040839    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:19.174032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.174006    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:19.460906 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.460861    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"]
Sep 08 14:34:19.461299 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.461283    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy"
Sep 08 14:34:19.461363 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.461300    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy"
Sep 08 14:34:19.461363 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.461315    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4" containerName="payload-pre-processing"
Sep 08 14:34:19.461363 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.461321    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4" containerName="payload-pre-processing"
Sep 08 14:34:19.461363 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.461330    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy"
Sep 08 14:34:19.461363 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.461335    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy"
Sep 08 14:34:19.461611 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.461411    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy"
Sep 08 14:34:19.461611 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.461420    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5728f139-a6a7-49ff-8bf8-1a0ff7d6e9c4" containerName="payload-pre-processing"
Sep 08 14:34:19.466004 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.465979    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.468653 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.468635    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a8fb2c38-openshift-default-dockercfg-kcczc\""
Sep 08 14:34:19.468758 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.468642    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 14:34:19.476078 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.476051    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"]
Sep 08 14:34:19.509971 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.509937    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:19.559887 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.559846    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5563b030-bc40-4449-b83e-377adc997461-istiod-ca-cert\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.560055 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.559900    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-envoy\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.560055 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.559996    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcbns\" (UniqueName: \"kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-kube-api-access-hcbns\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.560055 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.560027    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-certs\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.560055 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.560048    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-credential-socket\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.560189 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.560124    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-data\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.560189 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.560165    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5563b030-bc40-4449-b83e-377adc997461-istio-podinfo\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.560189 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.560182    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-istio-token\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.560317 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.560244    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-socket\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.661792 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.661749    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcbns\" (UniqueName: \"kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-kube-api-access-hcbns\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.662049 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.661808    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-certs\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.662049 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.661946    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-credential-socket\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.662049 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.661988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-data\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.662049 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.662038    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5563b030-bc40-4449-b83e-377adc997461-istio-podinfo\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.662335 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.662066    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-istio-token\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.662335 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.662094    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-socket\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.662335 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.662169    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5563b030-bc40-4449-b83e-377adc997461-istiod-ca-cert\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.662499 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.662292    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-envoy\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.662499 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.662413    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-certs\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.662499 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.662430    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-credential-socket\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.662702 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.662678    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-data\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.663127 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.663104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-socket\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.663323 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.663285    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5563b030-bc40-4449-b83e-377adc997461-istiod-ca-cert\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.664894 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.664871    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5563b030-bc40-4449-b83e-377adc997461-istio-podinfo\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.664984 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.664954    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-envoy\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.673239 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.673197    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-istio-token\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.673600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.673582    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcbns\" (UniqueName: \"kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-kube-api-access-hcbns\") pod \"e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") " pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:19.708339 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.708304    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:19.777806 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:19.777776    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:20.053132 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:20.053105    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"]
Sep 08 14:34:20.055097 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:34:20.055072    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5563b030_bc40_4449_b83e_377adc997461.slice/crio-ed049e6ff38f0974e622fe74dbedc580d2a71a86a812f73c4a95c964b9df90c9 WatchSource:0}: Error finding container ed049e6ff38f0974e622fe74dbedc580d2a71a86a812f73c4a95c964b9df90c9: Status 404 returned error can't find the container with id ed049e6ff38f0974e622fe74dbedc580d2a71a86a812f73c4a95c964b9df90c9
Sep 08 14:34:20.057257 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:20.057226    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:34:20.057325 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:20.057291    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:34:20.057360 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:20.057327    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:34:20.262140 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:20.262102    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" event={"ID":"5563b030-bc40-4449-b83e-377adc997461","Type":"ContainerStarted","Data":"94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81"}
Sep 08 14:34:20.262319 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:20.262155    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" event={"ID":"5563b030-bc40-4449-b83e-377adc997461","Type":"ContainerStarted","Data":"ed049e6ff38f0974e622fe74dbedc580d2a71a86a812f73c4a95c964b9df90c9"}
Sep 08 14:34:20.281481 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:20.281435    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" podStartSLOduration=1.28141864 podStartE2EDuration="1.28141864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:20.280602468 +0000 UTC m=+1075.511533666" watchObservedRunningTime="2026-09-08 14:34:20.28141864 +0000 UTC m=+1075.512349830"
Sep 08 14:34:20.456768 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:20.456685    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:20.778670 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:20.778634    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:21.783903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:21.783874    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:22.273346 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:22.273316    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:22.274357 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:22.274340    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:22.345250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:22.345217    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:22.442969 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:22.442935    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:23.672842 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:23.672813    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:23.736014 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:23.735976    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:23.849655 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:23.849619    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:34:23.890605 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:23.890573    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:24.179272 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:24.179239    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:24.276847 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:24.276812    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:36.658669 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:36.658623    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:34:37.961227 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:37.960317    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"]
Sep 08 14:34:37.961227 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:37.961118    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3c4f655-3238-4f74-bed2-906722d440a0" containerName="istio-proxy"
Sep 08 14:34:37.973230 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:37.969932    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:37.979238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:37.976076    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-a76115\"/\"noten-model-a76115-kserve-self-signed-certs\""
Sep 08 14:34:37.979238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:37.976395    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-a76115\"/\"default-dockercfg-5pxjv\""
Sep 08 14:34:37.979238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:37.976605    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-a76115\"/\"kube-root-ca.crt\""
Sep 08 14:34:37.979238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:37.977394    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-a76115\"/\"openshift-service-ca.crt\""
Sep 08 14:34:37.982581 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:37.982556    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"]
Sep 08 14:34:37.983958 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:37.983929    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:38.144672 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.144622    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-dshm\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.144932 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.144907    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf76m\" (UniqueName: \"kubernetes.io/projected/a10f7651-ec56-4420-af9e-9eae7575a895-kube-api-access-bf76m\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.145100 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.145083    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-tmp-dir\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.145297 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.145279    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.145419 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.145404    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-home\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.145563 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.145549    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-model-cache\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.247033 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.246982    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-model-cache\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.247225 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.247099    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-dshm\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.247600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.247577    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf76m\" (UniqueName: \"kubernetes.io/projected/a10f7651-ec56-4420-af9e-9eae7575a895-kube-api-access-bf76m\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.247718 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.247655    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-tmp-dir\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.247793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.247728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.247793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.247760    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-home\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.248267 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.248117    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-home\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.248642 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.248593    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-tmp-dir\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.248993 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.248974    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-model-cache\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.252881 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.252857    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-dshm\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.253123 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.253096    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.257976 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.257952    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf76m\" (UniqueName: \"kubernetes.io/projected/a10f7651-ec56-4420-af9e-9eae7575a895-kube-api-access-bf76m\") pod \"noten-model-a76115-kserve-7b97fbb4c5-tkx46\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") " pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.289216 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.289172    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:38.565084 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:38.565055    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"]
Sep 08 14:34:38.566585 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:34:38.566559    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda10f7651_ec56_4420_af9e_9eae7575a895.slice/crio-0f68662f007cea54d3806b4d79cab2cdd880945a01341ccdf2bcce073d41df5a WatchSource:0}: Error finding container 0f68662f007cea54d3806b4d79cab2cdd880945a01341ccdf2bcce073d41df5a: Status 404 returned error can't find the container with id 0f68662f007cea54d3806b4d79cab2cdd880945a01341ccdf2bcce073d41df5a
Sep 08 14:34:39.363568 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:39.363529    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46" event={"ID":"a10f7651-ec56-4420-af9e-9eae7575a895","Type":"ContainerStarted","Data":"d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422"}
Sep 08 14:34:39.363981 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:39.363576    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46" event={"ID":"a10f7651-ec56-4420-af9e-9eae7575a895","Type":"ContainerStarted","Data":"0f68662f007cea54d3806b4d79cab2cdd880945a01341ccdf2bcce073d41df5a"}
Sep 08 14:34:39.388887 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:39.388831    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46" podStartSLOduration=2.388814918 podStartE2EDuration="2.388814918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:39.387474939 +0000 UTC m=+1094.618406135" watchObservedRunningTime="2026-09-08 14:34:39.388814918 +0000 UTC m=+1094.619746158"
Sep 08 14:34:39.967823 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:39.967790    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:42.378895 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:42.378862    2558 generic.go:356] "Generic (PLEG): container finished" podID="5563b030-bc40-4449-b83e-377adc997461" containerID="94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81" exitCode=0
Sep 08 14:34:42.379346 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:42.378933    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" event={"ID":"5563b030-bc40-4449-b83e-377adc997461","Type":"ContainerDied","Data":"94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81"}
Sep 08 14:34:42.379405 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:42.379371    2558 scope.go:117] "RemoveContainer" containerID="94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81"
Sep 08 14:34:42.380438 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:42.380409    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:34:42.380497 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:42.380481    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:34:42.380533 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:42.380513    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:34:43.386633 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:43.386594    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" event={"ID":"5563b030-bc40-4449-b83e-377adc997461","Type":"ContainerStarted","Data":"bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749"}
Sep 08 14:34:43.779028 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:43.778979    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:43.779267 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:43.779243    2558 patch_prober.go:28] interesting pod/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 08 14:34:43.779325 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:43.779293    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 08 14:34:44.779221 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:44.779174    2558 patch_prober.go:28] interesting pod/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 08 14:34:44.779635 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:44.779274    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 08 14:34:45.778158 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:45.778117    2558 patch_prober.go:28] interesting pod/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 08 14:34:45.778349 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:45.778182    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 08 14:34:46.778393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:46.778353    2558 patch_prober.go:28] interesting pod/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 08 14:34:46.778776 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:46.778417    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 08 14:34:47.380847 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:47.380813    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:47.481141 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:47.481110    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:47.778423 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:47.778383    2558 patch_prober.go:28] interesting pod/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 08 14:34:47.778800 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:47.778440    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 08 14:34:48.172714 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.172612    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:48.288621 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.288587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:48.289337 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.289314    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:48.289544 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.289525    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:48.301648 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.301615    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"]
Sep 08 14:34:48.303502 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.303481    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:48.396859 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.396815    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:48.421384 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.421357    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:34:48.570773 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.570733    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:48.672745 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.672708    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:48.778669 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.778627    2558 patch_prober.go:28] interesting pod/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 08 14:34:48.779037 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:48.778714    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 08 14:34:49.156793 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:49.156756    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"]
Sep 08 14:34:49.165913 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:49.165887    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:49.305129 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:49.305098    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:49.474508 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:49.474413    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-self-signed-certs: secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:34:49.474508 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:49.474481    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs podName:a10f7651-ec56-4420-af9e-9eae7575a895 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:49.974465809 +0000 UTC m=+1105.205396985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs") pod "noten-model-a76115-kserve-7b97fbb4c5-tkx46" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895") : secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:34:49.541363 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:49.541328    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:49.642622 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:49.642589    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:49.781993 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:49.781964    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:49.782386 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:49.782245    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:49.782927 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:49.782911    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:34:49.980623 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:49.980583    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-self-signed-certs: secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:34:49.980796 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:49.980657    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs podName:a10f7651-ec56-4420-af9e-9eae7575a895 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:50.980641932 +0000 UTC m=+1106.211573106 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs") pod "noten-model-a76115-kserve-7b97fbb4c5-tkx46" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895") : secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:34:50.991091 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:50.991032    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-self-signed-certs: secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:34:50.991523 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:50.991126    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs podName:a10f7651-ec56-4420-af9e-9eae7575a895 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:52.991109585 +0000 UTC m=+1108.222040766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs") pod "noten-model-a76115-kserve-7b97fbb4c5-tkx46" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895") : secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:34:51.554736 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:51.554702    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:34:53.009010 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:53.008978    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-self-signed-certs: secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:34:53.009391 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:53.009053    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs podName:a10f7651-ec56-4420-af9e-9eae7575a895 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:57.009037235 +0000 UTC m=+1112.239968410 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs") pod "noten-model-a76115-kserve-7b97fbb4c5-tkx46" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895") : secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:34:54.723522 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:54.723485    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:54.824075 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:54.824025    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:55.199387 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:55.199349    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:57.051614 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:57.051568    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-self-signed-certs: secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:34:57.052012 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:34:57.051656    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs podName:a10f7651-ec56-4420-af9e-9eae7575a895 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:05.051640789 +0000 UTC m=+1120.282571965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs") pod "noten-model-a76115-kserve-7b97fbb4c5-tkx46" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895") : secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:34:58.556003 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:58.555973    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:59.349385 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:59.349348    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:59.526089 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:59.526051    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:59.594912 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:59.594884    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:59.833714 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:34:59.833680    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:01.359385 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:01.359345    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:35:03.321334 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.321295    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz" podUID="44f13dae-ddb8-48fd-a480-1a813baf3b2d" containerName="main" containerID="cri-o://eb9cc82c8a80185ff2aadd835b54e516f38ba026b1bc42c7defbb615a6ea37f8" gracePeriod=45
Sep 08 14:35:03.492819 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.492786    2558 generic.go:356] "Generic (PLEG): container finished" podID="44f13dae-ddb8-48fd-a480-1a813baf3b2d" containerID="eb9cc82c8a80185ff2aadd835b54e516f38ba026b1bc42c7defbb615a6ea37f8" exitCode=0
Sep 08 14:35:03.492965 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.492853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz" event={"ID":"44f13dae-ddb8-48fd-a480-1a813baf3b2d","Type":"ContainerDied","Data":"eb9cc82c8a80185ff2aadd835b54e516f38ba026b1bc42c7defbb615a6ea37f8"}
Sep 08 14:35:03.578593 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.578534    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:35:03.721628 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.721592    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-home\") pod \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") "
Sep 08 14:35:03.721628 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.721628    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-model-cache\") pod \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") "
Sep 08 14:35:03.721847 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.721685    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tls-certs\") pod \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") "
Sep 08 14:35:03.721847 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.721745    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2knf\" (UniqueName: \"kubernetes.io/projected/44f13dae-ddb8-48fd-a480-1a813baf3b2d-kube-api-access-p2knf\") pod \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") "
Sep 08 14:35:03.721847 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.721778    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-dshm\") pod \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") "
Sep 08 14:35:03.721847 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.721808    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tmp-dir\") pod \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\" (UID: \"44f13dae-ddb8-48fd-a480-1a813baf3b2d\") "
Sep 08 14:35:03.722063 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.721920    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-home" (OuterVolumeSpecName: "home") pod "44f13dae-ddb8-48fd-a480-1a813baf3b2d" (UID: "44f13dae-ddb8-48fd-a480-1a813baf3b2d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:03.722063 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.721936    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-model-cache" (OuterVolumeSpecName: "model-cache") pod "44f13dae-ddb8-48fd-a480-1a813baf3b2d" (UID: "44f13dae-ddb8-48fd-a480-1a813baf3b2d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:03.722174 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.722124    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-home\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:03.722174 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.722127    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "44f13dae-ddb8-48fd-a480-1a813baf3b2d" (UID: "44f13dae-ddb8-48fd-a480-1a813baf3b2d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:03.722174 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.722144    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-model-cache\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:03.723917 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.723884    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-dshm" (OuterVolumeSpecName: "dshm") pod "44f13dae-ddb8-48fd-a480-1a813baf3b2d" (UID: "44f13dae-ddb8-48fd-a480-1a813baf3b2d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:03.724029 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.723953    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "44f13dae-ddb8-48fd-a480-1a813baf3b2d" (UID: "44f13dae-ddb8-48fd-a480-1a813baf3b2d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:35:03.724029 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.723975    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44f13dae-ddb8-48fd-a480-1a813baf3b2d-kube-api-access-p2knf" (OuterVolumeSpecName: "kube-api-access-p2knf") pod "44f13dae-ddb8-48fd-a480-1a813baf3b2d" (UID: "44f13dae-ddb8-48fd-a480-1a813baf3b2d"). InnerVolumeSpecName "kube-api-access-p2knf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:35:03.822994 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.822957    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tls-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:03.822994 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.822987    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2knf\" (UniqueName: \"kubernetes.io/projected/44f13dae-ddb8-48fd-a480-1a813baf3b2d-kube-api-access-p2knf\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:03.822994 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.822999    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-dshm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:03.823239 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:03.823011    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f13dae-ddb8-48fd-a480-1a813baf3b2d-tmp-dir\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:04.499466 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:04.499437    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"
Sep 08 14:35:04.499928 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:04.499476    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz" event={"ID":"44f13dae-ddb8-48fd-a480-1a813baf3b2d","Type":"ContainerDied","Data":"32b7ecffb648265e7d6f51bf1889441da312b4c6b304ec98189b5b21422cf5df"}
Sep 08 14:35:04.499928 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:04.499527    2558 scope.go:117] "RemoveContainer" containerID="eb9cc82c8a80185ff2aadd835b54e516f38ba026b1bc42c7defbb615a6ea37f8"
Sep 08 14:35:04.522370 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:04.522340    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"]
Sep 08 14:35:04.526746 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:04.526726    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-2a2de3/test-model-2a2de3-kserve-59656675bc-jc8lz"]
Sep 08 14:35:05.138352 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:35:05.138319    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-self-signed-certs: secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:35:05.138523 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:35:05.138387    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs podName:a10f7651-ec56-4420-af9e-9eae7575a895 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:21.138372564 +0000 UTC m=+1136.369303740 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs") pod "noten-model-a76115-kserve-7b97fbb4c5-tkx46" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895") : secret "noten-model-a76115-kserve-self-signed-certs" not found
Sep 08 14:35:05.248258 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.248191    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:05.287262 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.287185    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44f13dae-ddb8-48fd-a480-1a813baf3b2d" path="/var/lib/kubelet/pods/44f13dae-ddb8-48fd-a480-1a813baf3b2d/volumes"
Sep 08 14:35:05.444944 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.444866    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46" podUID="a10f7651-ec56-4420-af9e-9eae7575a895" containerName="main" containerID="cri-o://d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422" gracePeriod=45
Sep 08 14:35:05.701666 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.701612    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:35:05.845378 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.845342    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bf76m\" (UniqueName: \"kubernetes.io/projected/a10f7651-ec56-4420-af9e-9eae7575a895-kube-api-access-bf76m\") pod \"a10f7651-ec56-4420-af9e-9eae7575a895\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") "
Sep 08 14:35:05.845543 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.845411    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-tmp-dir\") pod \"a10f7651-ec56-4420-af9e-9eae7575a895\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") "
Sep 08 14:35:05.845543 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.845442    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs\") pod \"a10f7651-ec56-4420-af9e-9eae7575a895\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") "
Sep 08 14:35:05.845543 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.845471    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-model-cache\") pod \"a10f7651-ec56-4420-af9e-9eae7575a895\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") "
Sep 08 14:35:05.845543 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.845524    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-dshm\") pod \"a10f7651-ec56-4420-af9e-9eae7575a895\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") "
Sep 08 14:35:05.845738 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.845565    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-home\") pod \"a10f7651-ec56-4420-af9e-9eae7575a895\" (UID: \"a10f7651-ec56-4420-af9e-9eae7575a895\") "
Sep 08 14:35:05.845738 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.845723    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a10f7651-ec56-4420-af9e-9eae7575a895" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:05.845839 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.845771    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-model-cache" (OuterVolumeSpecName: "model-cache") pod "a10f7651-ec56-4420-af9e-9eae7575a895" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:05.845895 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.845859    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-home" (OuterVolumeSpecName: "home") pod "a10f7651-ec56-4420-af9e-9eae7575a895" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:05.846172 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.846139    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-home\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:05.846172 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.846163    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-tmp-dir\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:05.846172 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.846177    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-model-cache\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:05.847675 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.847651    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a10f7651-ec56-4420-af9e-9eae7575a895-kube-api-access-bf76m" (OuterVolumeSpecName: "kube-api-access-bf76m") pod "a10f7651-ec56-4420-af9e-9eae7575a895" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895"). InnerVolumeSpecName "kube-api-access-bf76m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:35:05.847758 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.847721    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a10f7651-ec56-4420-af9e-9eae7575a895" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:35:05.847758 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.847740    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-dshm" (OuterVolumeSpecName: "dshm") pod "a10f7651-ec56-4420-af9e-9eae7575a895" (UID: "a10f7651-ec56-4420-af9e-9eae7575a895"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:05.947618 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.947581    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf76m\" (UniqueName: \"kubernetes.io/projected/a10f7651-ec56-4420-af9e-9eae7575a895-kube-api-access-bf76m\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:05.947781 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.947621    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10f7651-ec56-4420-af9e-9eae7575a895-tls-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:05.947781 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.947638    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10f7651-ec56-4420-af9e-9eae7575a895-dshm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:05.982810 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:05.982771    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:06.057450 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.057415    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:06.118797 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.118768    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:06.189525 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.189490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:06.251633 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.251557    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:06.512081 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.511996    2558 generic.go:356] "Generic (PLEG): container finished" podID="a10f7651-ec56-4420-af9e-9eae7575a895" containerID="d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422" exitCode=0
Sep 08 14:35:06.512081 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.512045    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"
Sep 08 14:35:06.512304 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.512088    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46" event={"ID":"a10f7651-ec56-4420-af9e-9eae7575a895","Type":"ContainerDied","Data":"d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422"}
Sep 08 14:35:06.512304 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.512136    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46" event={"ID":"a10f7651-ec56-4420-af9e-9eae7575a895","Type":"ContainerDied","Data":"0f68662f007cea54d3806b4d79cab2cdd880945a01341ccdf2bcce073d41df5a"}
Sep 08 14:35:06.512304 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.512158    2558 scope.go:117] "RemoveContainer" containerID="d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422"
Sep 08 14:35:06.521436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.521420    2558 scope.go:117] "RemoveContainer" containerID="d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422"
Sep 08 14:35:06.521718 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:35:06.521697    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422\": container with ID starting with d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422 not found: ID does not exist" containerID="d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422"
Sep 08 14:35:06.521766 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.521726    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422"} err="failed to get container status \"d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422\": rpc error: code = NotFound desc = could not find container \"d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422\": container with ID starting with d2ebdcc5cff0ff6945c1a6e3464059022ccd99663f1af17621bd8f6e90c96422 not found: ID does not exist"
Sep 08 14:35:06.534543 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.534519    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"]
Sep 08 14:35:06.538616 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.538595    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-a76115/noten-model-a76115-kserve-7b97fbb4c5-tkx46"]
Sep 08 14:35:06.588431 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.588381    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:06.948013 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:06.947924    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:07.048493 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:07.048460    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:07.236522 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:07.236489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:07.286711 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:07.286681    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a10f7651-ec56-4420-af9e-9eae7575a895" path="/var/lib/kubelet/pods/a10f7651-ec56-4420-af9e-9eae7575a895/volumes"
Sep 08 14:35:07.801933 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:07.801894    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:10.755678 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:10.755638    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:35:12.747858 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:12.747819    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:12.811430 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:12.811394    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:13.017981 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:13.017903    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"]
Sep 08 14:35:13.018182 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:13.018155    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy" containerID="cri-o://bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749" gracePeriod=30
Sep 08 14:35:13.053891 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:13.053856    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:13.394134 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:13.394047    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:13.496781 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:13.496743    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:13.624519 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:13.624485    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:18.147403 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.147362    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:18.174532 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.174491    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"]
Sep 08 14:35:18.174851 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.174822    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz" podUID="fd458ea6-161b-489e-b07d-905be3ef51eb" containerName="payload-processing" containerID="cri-o://7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035" gracePeriod=30
Sep 08 14:35:18.182037 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.182002    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"]
Sep 08 14:35:18.182536 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.182504    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s" podUID="b66ead7b-a5c7-48c7-87ed-502eb4507230" containerName="payload-pre-processing" containerID="cri-o://ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8" gracePeriod=30
Sep 08 14:35:18.216955 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.216921    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:18.306842 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.306816    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:35:18.330509 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.330433    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:18.362852 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.362824    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcbns\" (UniqueName: \"kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-kube-api-access-hcbns\") pod \"5563b030-bc40-4449-b83e-377adc997461\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") "
Sep 08 14:35:18.362987 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.362901    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-certs\") pod \"5563b030-bc40-4449-b83e-377adc997461\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") "
Sep 08 14:35:18.362987 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.362921    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5563b030-bc40-4449-b83e-377adc997461-istio-podinfo\") pod \"5563b030-bc40-4449-b83e-377adc997461\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") "
Sep 08 14:35:18.362987 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.362948    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-socket\") pod \"5563b030-bc40-4449-b83e-377adc997461\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") "
Sep 08 14:35:18.362987 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.362965    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-data\") pod \"5563b030-bc40-4449-b83e-377adc997461\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") "
Sep 08 14:35:18.363122 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.363022    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-envoy\") pod \"5563b030-bc40-4449-b83e-377adc997461\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") "
Sep 08 14:35:18.363122 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.363044    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5563b030-bc40-4449-b83e-377adc997461-istiod-ca-cert\") pod \"5563b030-bc40-4449-b83e-377adc997461\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") "
Sep 08 14:35:18.363122 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.363058    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-istio-token\") pod \"5563b030-bc40-4449-b83e-377adc997461\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") "
Sep 08 14:35:18.363122 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.363083    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-credential-socket\") pod \"5563b030-bc40-4449-b83e-377adc997461\" (UID: \"5563b030-bc40-4449-b83e-377adc997461\") "
Sep 08 14:35:18.364200 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.363460    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5563b030-bc40-4449-b83e-377adc997461" (UID: "5563b030-bc40-4449-b83e-377adc997461"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:18.364200 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.363871    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-data" (OuterVolumeSpecName: "istio-data") pod "5563b030-bc40-4449-b83e-377adc997461" (UID: "5563b030-bc40-4449-b83e-377adc997461"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:18.364200 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.363965    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5563b030-bc40-4449-b83e-377adc997461" (UID: "5563b030-bc40-4449-b83e-377adc997461"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:18.364470 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.364336    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5563b030-bc40-4449-b83e-377adc997461" (UID: "5563b030-bc40-4449-b83e-377adc997461"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:18.364616 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.364592    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5563b030-bc40-4449-b83e-377adc997461-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5563b030-bc40-4449-b83e-377adc997461" (UID: "5563b030-bc40-4449-b83e-377adc997461"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:35:18.366048 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.366019    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-kube-api-access-hcbns" (OuterVolumeSpecName: "kube-api-access-hcbns") pod "5563b030-bc40-4449-b83e-377adc997461" (UID: "5563b030-bc40-4449-b83e-377adc997461"). InnerVolumeSpecName "kube-api-access-hcbns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:35:18.367556 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.367524    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5563b030-bc40-4449-b83e-377adc997461" (UID: "5563b030-bc40-4449-b83e-377adc997461"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:35:18.367650 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.367565    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-istio-token" (OuterVolumeSpecName: "istio-token") pod "5563b030-bc40-4449-b83e-377adc997461" (UID: "5563b030-bc40-4449-b83e-377adc997461"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:35:18.367712 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.367662    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5563b030-bc40-4449-b83e-377adc997461-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5563b030-bc40-4449-b83e-377adc997461" (UID: "5563b030-bc40-4449-b83e-377adc997461"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:35:18.428745 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.428712    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:18.445903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.445879    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:35:18.451719 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.451698    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:35:18.464518 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.464491    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd458ea6-161b-489e-b07d-905be3ef51eb-plugins-config-volume\") pod \"fd458ea6-161b-489e-b07d-905be3ef51eb\" (UID: \"fd458ea6-161b-489e-b07d-905be3ef51eb\") "
Sep 08 14:35:18.464621 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.464601    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fm98\" (UniqueName: \"kubernetes.io/projected/fd458ea6-161b-489e-b07d-905be3ef51eb-kube-api-access-6fm98\") pod \"fd458ea6-161b-489e-b07d-905be3ef51eb\" (UID: \"fd458ea6-161b-489e-b07d-905be3ef51eb\") "
Sep 08 14:35:18.464684 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.464657    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b66ead7b-a5c7-48c7-87ed-502eb4507230-plugins-config-volume\") pod \"b66ead7b-a5c7-48c7-87ed-502eb4507230\" (UID: \"b66ead7b-a5c7-48c7-87ed-502eb4507230\") "
Sep 08 14:35:18.464740 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.464725    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs857\" (UniqueName: \"kubernetes.io/projected/b66ead7b-a5c7-48c7-87ed-502eb4507230-kube-api-access-bs857\") pod \"b66ead7b-a5c7-48c7-87ed-502eb4507230\" (UID: \"b66ead7b-a5c7-48c7-87ed-502eb4507230\") "
Sep 08 14:35:18.464924 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.464899    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd458ea6-161b-489e-b07d-905be3ef51eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd458ea6-161b-489e-b07d-905be3ef51eb" (UID: "fd458ea6-161b-489e-b07d-905be3ef51eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:35:18.465068 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.465049    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5563b030-bc40-4449-b83e-377adc997461-istiod-ca-cert\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.465140 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.465074    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-istio-token\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.465140 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.465088    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-credential-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.465140 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.465100    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd458ea6-161b-489e-b07d-905be3ef51eb-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.465332 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.465147    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcbns\" (UniqueName: \"kubernetes.io/projected/5563b030-bc40-4449-b83e-377adc997461-kube-api-access-hcbns\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.465332 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.465173    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.465332 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.465187    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5563b030-bc40-4449-b83e-377adc997461-istio-podinfo\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.465332 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.465199    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-workload-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.465332 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.465231    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-data\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.465332 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.465245    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5563b030-bc40-4449-b83e-377adc997461-istio-envoy\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.465332 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.465322    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b66ead7b-a5c7-48c7-87ed-502eb4507230-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b66ead7b-a5c7-48c7-87ed-502eb4507230" (UID: "b66ead7b-a5c7-48c7-87ed-502eb4507230"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:35:18.467317 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.467288    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd458ea6-161b-489e-b07d-905be3ef51eb-kube-api-access-6fm98" (OuterVolumeSpecName: "kube-api-access-6fm98") pod "fd458ea6-161b-489e-b07d-905be3ef51eb" (UID: "fd458ea6-161b-489e-b07d-905be3ef51eb"). InnerVolumeSpecName "kube-api-access-6fm98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:35:18.467432 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.467344    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b66ead7b-a5c7-48c7-87ed-502eb4507230-kube-api-access-bs857" (OuterVolumeSpecName: "kube-api-access-bs857") pod "b66ead7b-a5c7-48c7-87ed-502eb4507230" (UID: "b66ead7b-a5c7-48c7-87ed-502eb4507230"). InnerVolumeSpecName "kube-api-access-bs857". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:35:18.566008 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.565928    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fm98\" (UniqueName: \"kubernetes.io/projected/fd458ea6-161b-489e-b07d-905be3ef51eb-kube-api-access-6fm98\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.566008 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.565954    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b66ead7b-a5c7-48c7-87ed-502eb4507230-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.566008 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.565963    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs857\" (UniqueName: \"kubernetes.io/projected/b66ead7b-a5c7-48c7-87ed-502eb4507230-kube-api-access-bs857\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:35:18.579412 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.579381    2558 generic.go:356] "Generic (PLEG): container finished" podID="5563b030-bc40-4449-b83e-377adc997461" containerID="bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749" exitCode=0
Sep 08 14:35:18.579573 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.579432    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"
Sep 08 14:35:18.579573 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.579465    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" event={"ID":"5563b030-bc40-4449-b83e-377adc997461","Type":"ContainerDied","Data":"bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749"}
Sep 08 14:35:18.579573 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.579529    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw" event={"ID":"5563b030-bc40-4449-b83e-377adc997461","Type":"ContainerDied","Data":"ed049e6ff38f0974e622fe74dbedc580d2a71a86a812f73c4a95c964b9df90c9"}
Sep 08 14:35:18.579573 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.579561    2558 scope.go:117] "RemoveContainer" containerID="bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749"
Sep 08 14:35:18.580887 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.580861    2558 generic.go:356] "Generic (PLEG): container finished" podID="fd458ea6-161b-489e-b07d-905be3ef51eb" containerID="7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035" exitCode=0
Sep 08 14:35:18.580992 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.580952    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"
Sep 08 14:35:18.580992 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.580965    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz" event={"ID":"fd458ea6-161b-489e-b07d-905be3ef51eb","Type":"ContainerDied","Data":"7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035"}
Sep 08 14:35:18.581087 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.581002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz" event={"ID":"fd458ea6-161b-489e-b07d-905be3ef51eb","Type":"ContainerDied","Data":"00e5ee9b186ef23ecccebc8dbe1a6c2943967051ff99d25bf49d5d9a9082a8a4"}
Sep 08 14:35:18.582927 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.582904    2558 generic.go:356] "Generic (PLEG): container finished" podID="b66ead7b-a5c7-48c7-87ed-502eb4507230" containerID="ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8" exitCode=0
Sep 08 14:35:18.583085 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.582945    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s" event={"ID":"b66ead7b-a5c7-48c7-87ed-502eb4507230","Type":"ContainerDied","Data":"ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8"}
Sep 08 14:35:18.583187 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.583109    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s" event={"ID":"b66ead7b-a5c7-48c7-87ed-502eb4507230","Type":"ContainerDied","Data":"40efb743531f692a3d9498cfb0423097fbf6f3cc8ceaac420ceb1a8237af0fc3"}
Sep 08 14:35:18.583187 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.582961    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"
Sep 08 14:35:18.590517 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.590494    2558 scope.go:117] "RemoveContainer" containerID="94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81"
Sep 08 14:35:18.600690 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.600675    2558 scope.go:117] "RemoveContainer" containerID="bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749"
Sep 08 14:35:18.600981 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:35:18.600958    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749\": container with ID starting with bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749 not found: ID does not exist" containerID="bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749"
Sep 08 14:35:18.601054 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.600985    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749"} err="failed to get container status \"bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749\": rpc error: code = NotFound desc = could not find container \"bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749\": container with ID starting with bdd0030d2e9d31bfbffe9f3033ef9f9be27e0240e872754438c19cf9a0413749 not found: ID does not exist"
Sep 08 14:35:18.601054 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.601005    2558 scope.go:117] "RemoveContainer" containerID="94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81"
Sep 08 14:35:18.601273 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:35:18.601255    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81\": container with ID starting with 94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81 not found: ID does not exist" containerID="94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81"
Sep 08 14:35:18.601320 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.601278    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81"} err="failed to get container status \"94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81\": rpc error: code = NotFound desc = could not find container \"94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81\": container with ID starting with 94fc2d826f6f5e4a2bd10e6fe1e58308f52607c5f522e18b1581e1957d012e81 not found: ID does not exist"
Sep 08 14:35:18.601320 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.601292    2558 scope.go:117] "RemoveContainer" containerID="7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035"
Sep 08 14:35:18.608746 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.608697    2558 scope.go:117] "RemoveContainer" containerID="7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035"
Sep 08 14:35:18.609080 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:35:18.609053    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035\": container with ID starting with 7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035 not found: ID does not exist" containerID="7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035"
Sep 08 14:35:18.609158 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.609088    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035"} err="failed to get container status \"7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035\": rpc error: code = NotFound desc = could not find container \"7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035\": container with ID starting with 7b2b6860f0f561850bf2426d0427fc66c3f25c2373a29cc2644079c082bb2035 not found: ID does not exist"
Sep 08 14:35:18.609158 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.609109    2558 scope.go:117] "RemoveContainer" containerID="ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8"
Sep 08 14:35:18.612238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.612216    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"]
Sep 08 14:35:18.616242 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.616219    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a2de3-b66c64bfc-nlcgz"]
Sep 08 14:35:18.617279 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.617263    2558 scope.go:117] "RemoveContainer" containerID="ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8"
Sep 08 14:35:18.617528 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:35:18.617509    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8\": container with ID starting with ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8 not found: ID does not exist" containerID="ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8"
Sep 08 14:35:18.617593 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.617536    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8"} err="failed to get container status \"ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8\": rpc error: code = NotFound desc = could not find container \"ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8\": container with ID starting with ea5a03e7d4c7e1f1f2b709d2a6b48db52315186bf0f803bb480ffae326cce5e8 not found: ID does not exist"
Sep 08 14:35:18.629764 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.629744    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"]
Sep 08 14:35:18.632577 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.632556    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a8fb2c38-openshift-default-59d8c64596-nbbgw"]
Sep 08 14:35:18.652243 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.652198    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"]
Sep 08 14:35:18.659148 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.659129    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2a2de3-7bbcb8f68b-94f7s"]
Sep 08 14:35:18.841990 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:18.841897    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:19.286606 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:19.286572    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5563b030-bc40-4449-b83e-377adc997461" path="/var/lib/kubelet/pods/5563b030-bc40-4449-b83e-377adc997461/volumes"
Sep 08 14:35:19.286961 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:19.286947    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b66ead7b-a5c7-48c7-87ed-502eb4507230" path="/var/lib/kubelet/pods/b66ead7b-a5c7-48c7-87ed-502eb4507230/volumes"
Sep 08 14:35:19.287306 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:19.287292    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd458ea6-161b-489e-b07d-905be3ef51eb" path="/var/lib/kubelet/pods/fd458ea6-161b-489e-b07d-905be3ef51eb/volumes"
Sep 08 14:35:19.961293 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:19.961262    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:35:20.404484 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:20.404431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:20.511036 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:20.511001    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:20.611958 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:20.611927    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:20.748317 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:20.748287    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:24.756622 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:24.756583    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:24.820373 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:24.820339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:25.036105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:25.036032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:25.616101 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:25.616070    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:25.908832 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:25.908800    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:26.200640 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:26.200558    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:26.969809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:26.969776    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:27.072344 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:27.072309    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:27.900614 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:27.900583    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:28.173302 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.173042    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"]
Sep 08 14:35:28.174546 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.174519    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b66ead7b-a5c7-48c7-87ed-502eb4507230" containerName="payload-pre-processing"
Sep 08 14:35:28.174702 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.174688    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b66ead7b-a5c7-48c7-87ed-502eb4507230" containerName="payload-pre-processing"
Sep 08 14:35:28.174795 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.174784    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd458ea6-161b-489e-b07d-905be3ef51eb" containerName="payload-processing"
Sep 08 14:35:28.174880 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.174869    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd458ea6-161b-489e-b07d-905be3ef51eb" containerName="payload-processing"
Sep 08 14:35:28.174984 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.174973    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy"
Sep 08 14:35:28.175065 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175056    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy"
Sep 08 14:35:28.175155 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175145    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy"
Sep 08 14:35:28.175260 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175235    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy"
Sep 08 14:35:28.175393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175265    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f13dae-ddb8-48fd-a480-1a813baf3b2d" containerName="main"
Sep 08 14:35:28.175393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175274    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f13dae-ddb8-48fd-a480-1a813baf3b2d" containerName="main"
Sep 08 14:35:28.175393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175289    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a10f7651-ec56-4420-af9e-9eae7575a895" containerName="main"
Sep 08 14:35:28.175393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175297    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a10f7651-ec56-4420-af9e-9eae7575a895" containerName="main"
Sep 08 14:35:28.175571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175474    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f13dae-ddb8-48fd-a480-1a813baf3b2d" containerName="main"
Sep 08 14:35:28.175571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175495    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy"
Sep 08 14:35:28.175571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175515    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a10f7651-ec56-4420-af9e-9eae7575a895" containerName="main"
Sep 08 14:35:28.175571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175527    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b66ead7b-a5c7-48c7-87ed-502eb4507230" containerName="payload-pre-processing"
Sep 08 14:35:28.175571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.175543    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd458ea6-161b-489e-b07d-905be3ef51eb" containerName="payload-processing"
Sep 08 14:35:28.180723 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.180702    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.183515 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.183481    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-23a1d4-openshift-default-dockercfg-ptm9g\""
Sep 08 14:35:28.183790 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.183730    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 14:35:28.184638 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.184612    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"]
Sep 08 14:35:28.196860 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.196835    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:28.257847 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.257814    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-certs\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.257988 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.257858    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istiod-ca-cert\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.257988 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.257928    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztnnr\" (UniqueName: \"kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-kube-api-access-ztnnr\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.258092 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.258002    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-data\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.258092 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.258065    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-credential-socket\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.258165 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.258098    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-podinfo\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.258165 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.258119    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-envoy\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.258165 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.258147    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-socket\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.258309 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.258228    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-token\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359244 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359187    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztnnr\" (UniqueName: \"kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-kube-api-access-ztnnr\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359282    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-data\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359306    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-credential-socket\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359323    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-podinfo\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359347    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-envoy\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359386    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-socket\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359419    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-token\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359481    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-certs\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359526    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istiod-ca-cert\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359801 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359741    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-credential-socket\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359801 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359780    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-data\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359916 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359806    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-certs\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.359916 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.359854    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-socket\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.360105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.360087    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istiod-ca-cert\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.361712 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.361684    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-envoy\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.361836 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.361819    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-podinfo\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.367960 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.367931    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-token\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.368038 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.368021    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztnnr\" (UniqueName: \"kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-kube-api-access-ztnnr\") pod \"e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") " pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.457644 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.457555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:28.492325 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.492294    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:28.767867 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.767842    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"]
Sep 08 14:35:28.769843 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:35:28.769811    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07d5b5b_6a9b_44c5_bd00_fd9084282e66.slice/crio-656998680f16330ffafca0a6d30343d9d2de941a853ccb62426bbb0a31a1090b WatchSource:0}: Error finding container 656998680f16330ffafca0a6d30343d9d2de941a853ccb62426bbb0a31a1090b: Status 404 returned error can't find the container with id 656998680f16330ffafca0a6d30343d9d2de941a853ccb62426bbb0a31a1090b
Sep 08 14:35:28.772265 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.772192    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:35:28.772337 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.772322    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:35:28.772383 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:28.772366    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:35:29.332636 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:29.332607    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:29.431548 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:29.431516    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:29.638627 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:29.638544    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" event={"ID":"d07d5b5b-6a9b-44c5-bd00-fd9084282e66","Type":"ContainerStarted","Data":"87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90"}
Sep 08 14:35:29.638627 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:29.638598    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" event={"ID":"d07d5b5b-6a9b-44c5-bd00-fd9084282e66","Type":"ContainerStarted","Data":"656998680f16330ffafca0a6d30343d9d2de941a853ccb62426bbb0a31a1090b"}
Sep 08 14:35:29.663734 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:29.663682    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" podStartSLOduration=1.6636683030000001 podStartE2EDuration="1.663668303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:29.660922594 +0000 UTC m=+1144.891853788" watchObservedRunningTime="2026-09-08 14:35:29.663668303 +0000 UTC m=+1144.894599499"
Sep 08 14:35:30.493266 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:30.493227    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:30.498716 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:30.498683    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:30.588216 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:30.588173    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:30.643291 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:30.643259    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:30.644178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:30.644158    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:30.666771 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:30.666738    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:30.788525 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:30.788445    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:31.197703 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:31.197623    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:31.297751 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:31.297718    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:31.742842 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:31.742805    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:32.869371 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:32.869330    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:32.971324 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:32.971287    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:33.404402 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:33.404371    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:33.465247 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:33.465194    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:33.657508 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:33.657433    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:35:33.911880 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:33.911801    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:34.073416 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.073385    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:34.167338 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.167257    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:34.255259 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.255217    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"]
Sep 08 14:35:34.255662 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.255650    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5563b030-bc40-4449-b83e-377adc997461" containerName="istio-proxy"
Sep 08 14:35:34.259008 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.258991    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:35:34.261457 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.261431    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-23a1d4-dockercfg-xwvg9\""
Sep 08 14:35:34.261599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.261462    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-23a1d4\""
Sep 08 14:35:34.268502 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.268479    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"]
Sep 08 14:35:34.316281 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.316245    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d08211f-12b3-4cf7-9a22-26aa1bff471b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2\" (UID: \"8d08211f-12b3-4cf7-9a22-26aa1bff471b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:35:34.316427 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.316345    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlfb2\" (UniqueName: \"kubernetes.io/projected/8d08211f-12b3-4cf7-9a22-26aa1bff471b-kube-api-access-dlfb2\") pod \"payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2\" (UID: \"8d08211f-12b3-4cf7-9a22-26aa1bff471b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:35:34.417402 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.417325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d08211f-12b3-4cf7-9a22-26aa1bff471b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2\" (UID: \"8d08211f-12b3-4cf7-9a22-26aa1bff471b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:35:34.417547 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.417410    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlfb2\" (UniqueName: \"kubernetes.io/projected/8d08211f-12b3-4cf7-9a22-26aa1bff471b-kube-api-access-dlfb2\") pod \"payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2\" (UID: \"8d08211f-12b3-4cf7-9a22-26aa1bff471b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:35:34.417998 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.417976    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d08211f-12b3-4cf7-9a22-26aa1bff471b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2\" (UID: \"8d08211f-12b3-4cf7-9a22-26aa1bff471b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:35:34.425870 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.425842    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlfb2\" (UniqueName: \"kubernetes.io/projected/8d08211f-12b3-4cf7-9a22-26aa1bff471b-kube-api-access-dlfb2\") pod \"payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2\" (UID: \"8d08211f-12b3-4cf7-9a22-26aa1bff471b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:35:34.571221 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.571183    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:35:34.840041 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:34.840014    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"]
Sep 08 14:35:34.842976 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:35:34.842949    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d08211f_12b3_4cf7_9a22_26aa1bff471b.slice/crio-24225d01371087d31633f6caac4250577a0f3222820fdd29d5350824d340f893 WatchSource:0}: Error finding container 24225d01371087d31633f6caac4250577a0f3222820fdd29d5350824d340f893: Status 404 returned error can't find the container with id 24225d01371087d31633f6caac4250577a0f3222820fdd29d5350824d340f893
Sep 08 14:35:35.674166 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:35.674132    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2" event={"ID":"8d08211f-12b3-4cf7-9a22-26aa1bff471b","Type":"ContainerStarted","Data":"1b724d65871cf44ee4c913f0c957ba060b64928b8a5e44d4299f0f5a59dd5c72"}
Sep 08 14:35:35.674612 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:35.674179    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2" event={"ID":"8d08211f-12b3-4cf7-9a22-26aa1bff471b","Type":"ContainerStarted","Data":"24225d01371087d31633f6caac4250577a0f3222820fdd29d5350824d340f893"}
Sep 08 14:35:35.674612 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:35.674266    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:35:35.693382 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:35.693337    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2" podStartSLOduration=1.693322444 podStartE2EDuration="1.693322444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:35.692226867 +0000 UTC m=+1150.923158063" watchObservedRunningTime="2026-09-08 14:35:35.693322444 +0000 UTC m=+1150.924253640"
Sep 08 14:35:45.757921 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:45.757873    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:35:46.681126 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:46.681096    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:35:51.758504 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:51.758470    2558 generic.go:356] "Generic (PLEG): container finished" podID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerID="87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90" exitCode=0
Sep 08 14:35:51.758913 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:51.758548    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" event={"ID":"d07d5b5b-6a9b-44c5-bd00-fd9084282e66","Type":"ContainerDied","Data":"87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90"}
Sep 08 14:35:51.759050 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:51.759032    2558 scope.go:117] "RemoveContainer" containerID="87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90"
Sep 08 14:35:51.760156 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:51.760093    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:35:51.760234 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:51.760161    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:35:51.760234 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:51.760194    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:35:52.769097 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:52.769049    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" event={"ID":"d07d5b5b-6a9b-44c5-bd00-fd9084282e66","Type":"ContainerStarted","Data":"cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4"}
Sep 08 14:35:52.798901 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:52.798863    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:53.492653 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:53.492615    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:53.492877 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:53.492854    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 08 14:35:53.492926 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:53.492907    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 08 14:35:54.493263 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:54.493220    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 08 14:35:54.493639 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:54.493292    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 08 14:35:54.542922 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:54.542882    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:54.698692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:54.698650    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:54.799711 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:54.799620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:55.010008 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.009970    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"]
Sep 08 14:35:55.013765 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.013741    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.019110 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.018913    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-017dbd-openshift-default-dockercfg-p4mfn\""
Sep 08 14:35:55.026484 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.026453    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"]
Sep 08 14:35:55.038632 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.038592    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:55.116218 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.116089    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-certs\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.116218 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.116150    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-data\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.116218 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.116173    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-envoy\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.116511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.116230    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-podinfo\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.116511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.116296    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-credential-socket\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.116511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.116359    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxvfd\" (UniqueName: \"kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-kube-api-access-wxvfd\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.116511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.116446    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-socket\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.116511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.116466    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istiod-ca-cert\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.116775 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.116514    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-token\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217167 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217132    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-credential-socket\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217367 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217173    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxvfd\" (UniqueName: \"kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-kube-api-access-wxvfd\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217367 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217240    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-socket\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217367 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217260    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istiod-ca-cert\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217367 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217306    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-token\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217367 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217345    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-certs\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217608 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217367    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-data\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217608 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217393    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-envoy\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217608 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217430    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-podinfo\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217745 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217626    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-credential-socket\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217745 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217675    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-socket\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217745 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217727    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-certs\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.217870 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.217847    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-data\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.218032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.218015    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istiod-ca-cert\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.219695 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.219671    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-envoy\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.219838 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.219820    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-podinfo\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.227249 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.227200    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-token\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.227461 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.227440    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxvfd\" (UniqueName: \"kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-kube-api-access-wxvfd\") pod \"e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") " pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.228602 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.228582    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:55.330537 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.330497    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:55.493443 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.493405    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 08 14:35:55.493815 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.493478    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 08 14:35:55.625014 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.624984    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"]
Sep 08 14:35:55.626291 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:35:55.626264    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod519c6d5f_819a_469f_ab5f_267ac1d7cd7d.slice/crio-1434f423f5c150c310a63468a86d59156645de37b2f827330b332ecad7e48cf5 WatchSource:0}: Error finding container 1434f423f5c150c310a63468a86d59156645de37b2f827330b332ecad7e48cf5: Status 404 returned error can't find the container with id 1434f423f5c150c310a63468a86d59156645de37b2f827330b332ecad7e48cf5
Sep 08 14:35:55.628491 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.628457    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:35:55.628561 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.628521    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:35:55.628561 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.628552    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:35:55.785583 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.785543    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" event={"ID":"519c6d5f-819a-469f-ab5f-267ac1d7cd7d","Type":"ContainerStarted","Data":"1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8"}
Sep 08 14:35:55.785781 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.785600    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" event={"ID":"519c6d5f-819a-469f-ab5f-267ac1d7cd7d","Type":"ContainerStarted","Data":"1434f423f5c150c310a63468a86d59156645de37b2f827330b332ecad7e48cf5"}
Sep 08 14:35:55.807014 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:55.806602    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" podStartSLOduration=1.8065807409999999 podStartE2EDuration="1.806580741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:55.806193887 +0000 UTC m=+1171.037125085" watchObservedRunningTime="2026-09-08 14:35:55.806580741 +0000 UTC m=+1171.037511940"
Sep 08 14:35:56.331706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:56.331671    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:56.337221 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:56.337185    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:56.493527 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:56.493489    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 08 14:35:56.493912 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:56.493552    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 08 14:35:56.627025 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:56.626926    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:56.791144 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:56.791110    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:56.792092 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:56.792064    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:35:56.961423 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:56.961282    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:57.296508 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:57.296474    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:57.455035 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:57.454994    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:57.493054 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:57.493017    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 08 14:35:57.493295 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:57.493086    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 08 14:35:57.776743 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:57.776706    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:57.861310 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:57.861272    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:58.531759 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:58.531729    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:58.532030 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:58.532013    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:58.532809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:58.532788    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:35:58.931529 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:58.931447    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:59.031768 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.031736    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:59.183081 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.182986    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:59.510908 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.510874    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:59.578939 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.578904    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"]
Sep 08 14:35:59.581533 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.581502    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:59.582755 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.582735    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:35:59.585488 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.585464    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-017dbd\""
Sep 08 14:35:59.585585 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.585525    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-017dbd-dockercfg-njktf\""
Sep 08 14:35:59.595538 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.595514    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"]
Sep 08 14:35:59.658684 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.658651    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvcqs\" (UniqueName: \"kubernetes.io/projected/8fd21f5f-823a-4057-ac83-34865aaa3d1c-kube-api-access-lvcqs\") pod \"payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb\" (UID: \"8fd21f5f-823a-4057-ac83-34865aaa3d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:35:59.658843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.658721    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fd21f5f-823a-4057-ac83-34865aaa3d1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb\" (UID: \"8fd21f5f-823a-4057-ac83-34865aaa3d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:35:59.759420 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.759386    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fd21f5f-823a-4057-ac83-34865aaa3d1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb\" (UID: \"8fd21f5f-823a-4057-ac83-34865aaa3d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:35:59.759587 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.759473    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvcqs\" (UniqueName: \"kubernetes.io/projected/8fd21f5f-823a-4057-ac83-34865aaa3d1c-kube-api-access-lvcqs\") pod \"payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb\" (UID: \"8fd21f5f-823a-4057-ac83-34865aaa3d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:35:59.760070 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.760051    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fd21f5f-823a-4057-ac83-34865aaa3d1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb\" (UID: \"8fd21f5f-823a-4057-ac83-34865aaa3d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:35:59.768631 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.768565    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvcqs\" (UniqueName: \"kubernetes.io/projected/8fd21f5f-823a-4057-ac83-34865aaa3d1c-kube-api-access-lvcqs\") pod \"payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb\" (UID: \"8fd21f5f-823a-4057-ac83-34865aaa3d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:35:59.894334 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:35:59.894300    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:36:00.173585 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:00.173556    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"]
Sep 08 14:36:00.175647 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:36:00.175621    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd21f5f_823a_4057_ac83_34865aaa3d1c.slice/crio-54fa182e92bd89ece1dc327a978388373c0790bf31eb3bd623fe0f7ca7e9e9df WatchSource:0}: Error finding container 54fa182e92bd89ece1dc327a978388373c0790bf31eb3bd623fe0f7ca7e9e9df: Status 404 returned error can't find the container with id 54fa182e92bd89ece1dc327a978388373c0790bf31eb3bd623fe0f7ca7e9e9df
Sep 08 14:36:00.632308 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:00.632270    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:00.734610 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:00.734576    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:00.813905 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:00.813870    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb" event={"ID":"8fd21f5f-823a-4057-ac83-34865aaa3d1c","Type":"ContainerStarted","Data":"62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0"}
Sep 08 14:36:00.814115 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:00.813929    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:36:00.814115 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:00.813949    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb" event={"ID":"8fd21f5f-823a-4057-ac83-34865aaa3d1c","Type":"ContainerStarted","Data":"54fa182e92bd89ece1dc327a978388373c0790bf31eb3bd623fe0f7ca7e9e9df"}
Sep 08 14:36:00.831846 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:00.831799    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb" podStartSLOduration=1.8317847779999998 podStartE2EDuration="1.831784778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:36:00.830505677 +0000 UTC m=+1176.061436875" watchObservedRunningTime="2026-09-08 14:36:00.831784778 +0000 UTC m=+1176.062715974"
Sep 08 14:36:01.783492 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:01.783460    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:01.884103 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:01.884060    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:02.297791 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.297750    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:02.538706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.538674    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"]
Sep 08 14:36:02.542525 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.542503    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:36:02.545044 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.545022    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-34656b\""
Sep 08 14:36:02.545185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.545114    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-34656b-dockercfg-hdkgv\""
Sep 08 14:36:02.551478 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.551452    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"]
Sep 08 14:36:02.587708 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.587675    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba6c4672-5908-4b5d-a2eb-86ac24fee986-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6\" (UID: \"ba6c4672-5908-4b5d-a2eb-86ac24fee986\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:36:02.587858 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.587745    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfqtr\" (UniqueName: \"kubernetes.io/projected/ba6c4672-5908-4b5d-a2eb-86ac24fee986-kube-api-access-jfqtr\") pod \"payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6\" (UID: \"ba6c4672-5908-4b5d-a2eb-86ac24fee986\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:36:02.689067 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.689033    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba6c4672-5908-4b5d-a2eb-86ac24fee986-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6\" (UID: \"ba6c4672-5908-4b5d-a2eb-86ac24fee986\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:36:02.689067 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.689080    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfqtr\" (UniqueName: \"kubernetes.io/projected/ba6c4672-5908-4b5d-a2eb-86ac24fee986-kube-api-access-jfqtr\") pod \"payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6\" (UID: \"ba6c4672-5908-4b5d-a2eb-86ac24fee986\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:36:02.689965 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.689940    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba6c4672-5908-4b5d-a2eb-86ac24fee986-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6\" (UID: \"ba6c4672-5908-4b5d-a2eb-86ac24fee986\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:36:02.697309 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.697287    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfqtr\" (UniqueName: \"kubernetes.io/projected/ba6c4672-5908-4b5d-a2eb-86ac24fee986-kube-api-access-jfqtr\") pod \"payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6\" (UID: \"ba6c4672-5908-4b5d-a2eb-86ac24fee986\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:36:02.855080 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:02.855005    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:36:03.126734 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:03.126709    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"]
Sep 08 14:36:03.129228 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:36:03.129182    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba6c4672_5908_4b5d_a2eb_86ac24fee986.slice/crio-f80e0eee740136a855d1372fae33c9c521d8d3b1a06b8e3d87953e090518aaba WatchSource:0}: Error finding container f80e0eee740136a855d1372fae33c9c521d8d3b1a06b8e3d87953e090518aaba: Status 404 returned error can't find the container with id f80e0eee740136a855d1372fae33c9c521d8d3b1a06b8e3d87953e090518aaba
Sep 08 14:36:03.211893 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:03.211865    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:03.310357 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:03.310327    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:03.830541 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:03.830507    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6" event={"ID":"ba6c4672-5908-4b5d-a2eb-86ac24fee986","Type":"ContainerStarted","Data":"54fa355cc8034820d5f7a6d6fef232caca1aa18523d66d53a17bcd3619f5c521"}
Sep 08 14:36:03.830757 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:03.830565    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6" event={"ID":"ba6c4672-5908-4b5d-a2eb-86ac24fee986","Type":"ContainerStarted","Data":"f80e0eee740136a855d1372fae33c9c521d8d3b1a06b8e3d87953e090518aaba"}
Sep 08 14:36:03.830757 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:03.830592    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:36:03.847728 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:03.847588    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6" podStartSLOduration=1.847569542 podStartE2EDuration="1.847569542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:36:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:36:03.847289623 +0000 UTC m=+1179.078220867" watchObservedRunningTime="2026-09-08 14:36:03.847569542 +0000 UTC m=+1179.078500739"
Sep 08 14:36:05.263054 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:05.263013    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:36:11.820644 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:11.820611    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:36:14.837591 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:14.837554    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:36:16.911330 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:16.911294    2558 generic.go:356] "Generic (PLEG): container finished" podID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerID="1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8" exitCode=0
Sep 08 14:36:16.911785 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:16.911332    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" event={"ID":"519c6d5f-819a-469f-ab5f-267ac1d7cd7d","Type":"ContainerDied","Data":"1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8"}
Sep 08 14:36:16.911828 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:16.911796    2558 scope.go:117] "RemoveContainer" containerID="1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8"
Sep 08 14:36:16.912861 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:16.912829    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:36:16.912917 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:16.912902    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:36:16.912956 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:16.912935    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:36:17.918562 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:17.918530    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" event={"ID":"519c6d5f-819a-469f-ab5f-267ac1d7cd7d","Type":"ContainerStarted","Data":"67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c"}
Sep 08 14:36:18.331377 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:18.331340    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:36:18.331576 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:18.331556    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 08 14:36:18.331638 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:18.331620    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 08 14:36:19.331143 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:19.331104    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 08 14:36:19.331566 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:19.331164    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 08 14:36:19.977008 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:19.976972    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:20.331241 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:20.331177    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 08 14:36:20.331634 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:20.331294    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 08 14:36:21.331675 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:21.331637    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 08 14:36:21.332071 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:21.331691    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 08 14:36:22.331455 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:22.331421    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 08 14:36:22.331635 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:22.331475    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 08 14:36:22.829838 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:22.829806    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:22.930857 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:22.930818    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:23.335495 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:23.335459    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:36:23.335831 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:23.335807    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:36:23.336578 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:23.336560    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:36:25.393421 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:25.393391    2558 scope.go:117] "RemoveContainer" containerID="578d30bc9e1b4e0320bc53871ab08bf89daf109d11b9a6507a0835ad82f34570"
Sep 08 14:36:25.401330 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:25.401305    2558 scope.go:117] "RemoveContainer" containerID="a2821fa146393a0b4f6721cb0474036aadc91736fd41a48290f261d5302b8e68"
Sep 08 14:36:25.515346 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:25.515317    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:28.140075 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.140034    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"]
Sep 08 14:36:28.145433 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.145402    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.149120 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.149094    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-780944\"/\"auth-test-model-780944-kserve-self-signed-certs\""
Sep 08 14:36:28.149380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.149114    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-780944\"/\"openshift-service-ca.crt\""
Sep 08 14:36:28.149549 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.149129    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-780944\"/\"default-dockercfg-vdhn5\""
Sep 08 14:36:28.149657 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.149131    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-780944\"/\"kube-root-ca.crt\""
Sep 08 14:36:28.170228 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.153671    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"]
Sep 08 14:36:28.177231 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.177150    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:28.260662 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.260626    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-home\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.260825 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.260679    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-dshm\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.260825 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.260765    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/377f1853-1121-47c8-ac3b-9c97f1ec8463-tls-certs\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.260825 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.260799    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-model-cache\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.260930 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.260829    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs8bf\" (UniqueName: \"kubernetes.io/projected/377f1853-1121-47c8-ac3b-9c97f1ec8463-kube-api-access-qs8bf\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.260930 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.260851    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-tmp-dir\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.362052 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.362016    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-dshm\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.362247 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.362080    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/377f1853-1121-47c8-ac3b-9c97f1ec8463-tls-certs\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.362247 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.362117    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-model-cache\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.362247 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.362147    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs8bf\" (UniqueName: \"kubernetes.io/projected/377f1853-1121-47c8-ac3b-9c97f1ec8463-kube-api-access-qs8bf\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.362382 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.362271    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-tmp-dir\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.362436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.362419    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-home\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.362559 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.362536    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-model-cache\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.362715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.362698    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-tmp-dir\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.362782 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.362713    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-home\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.364404 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.364384    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-dshm\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.364652 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.364634    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/377f1853-1121-47c8-ac3b-9c97f1ec8463-tls-certs\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.370851 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.370827    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs8bf\" (UniqueName: \"kubernetes.io/projected/377f1853-1121-47c8-ac3b-9c97f1ec8463-kube-api-access-qs8bf\") pod \"auth-test-model-780944-kserve-5c8997794c-7ttns\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") " pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.480443 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.480359    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:28.752386 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.752359    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"]
Sep 08 14:36:28.754312 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:36:28.754284    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod377f1853_1121_47c8_ac3b_9c97f1ec8463.slice/crio-5f6e810990d94b14891bca80eb37ac0111544c014f9bed4346c74c20e9523725 WatchSource:0}: Error finding container 5f6e810990d94b14891bca80eb37ac0111544c014f9bed4346c74c20e9523725: Status 404 returned error can't find the container with id 5f6e810990d94b14891bca80eb37ac0111544c014f9bed4346c74c20e9523725
Sep 08 14:36:28.981639 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.981599    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns" event={"ID":"377f1853-1121-47c8-ac3b-9c97f1ec8463","Type":"ContainerStarted","Data":"f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3"}
Sep 08 14:36:28.981639 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:28.981645    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns" event={"ID":"377f1853-1121-47c8-ac3b-9c97f1ec8463","Type":"ContainerStarted","Data":"5f6e810990d94b14891bca80eb37ac0111544c014f9bed4346c74c20e9523725"}
Sep 08 14:36:29.002712 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:29.002609    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns" podStartSLOduration=1.002594127 podStartE2EDuration="1.002594127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:36:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:36:29.000719847 +0000 UTC m=+1204.231651045" watchObservedRunningTime="2026-09-08 14:36:29.002594127 +0000 UTC m=+1204.233525323"
Sep 08 14:36:31.362809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:31.362773    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:36:34.333953 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.333920    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:34.603759 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.603683    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"]
Sep 08 14:36:34.607744 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.607719    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.610443 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.610423    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-afb814fe-openshift-default-dockercfg-pxlc2\""
Sep 08 14:36:34.619022 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.619001    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"]
Sep 08 14:36:34.626470 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.626448    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-certs\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.626569 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.626479    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnddb\" (UniqueName: \"kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-kube-api-access-cnddb\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.626569 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.626507    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-credential-socket\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.626569 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.626540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-data\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.626703 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.626619    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-token\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.626703 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.626660    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-podinfo\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.626788 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.626712    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-envoy\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.626878 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.626855    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istiod-ca-cert\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.626936 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.626900    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-socket\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.636106 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.636086    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:34.728129 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.728089    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-token\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.728129 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.728132    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-podinfo\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.728408 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.728258    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-envoy\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.728408 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.728370    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istiod-ca-cert\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.728516 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.728413    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-socket\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.728516 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.728492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-certs\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.728620 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.728520    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnddb\" (UniqueName: \"kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-kube-api-access-cnddb\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.728620 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.728568    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-credential-socket\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.728746 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.728654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-data\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.728808 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.728785    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-socket\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.729022 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.728993    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-certs\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.729162 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.729056    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-credential-socket\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.729162 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.729072    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-data\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.729162 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.729123    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istiod-ca-cert\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.730558 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.730536    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-envoy\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.730771 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.730755    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-podinfo\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.736602 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.736580    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-token\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.737342 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.737324    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnddb\" (UniqueName: \"kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-kube-api-access-cnddb\") pod \"e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") " pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:34.920763 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:34.920679    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:35.197462 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:35.197423    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"]
Sep 08 14:36:35.199167 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:36:35.199136    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d489c4_e94c_45ed_82f8_3b0e2b389e9d.slice/crio-209e0b531843d0e5806b7052cc705160328240c2f71eb96e49d6259e2946bf26 WatchSource:0}: Error finding container 209e0b531843d0e5806b7052cc705160328240c2f71eb96e49d6259e2946bf26: Status 404 returned error can't find the container with id 209e0b531843d0e5806b7052cc705160328240c2f71eb96e49d6259e2946bf26
Sep 08 14:36:35.201500 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:35.201468    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:36:35.201568 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:35.201532    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:36:35.201568 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:35.201563    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:36:35.480987 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:35.480909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:36.019496 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:36.019450    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" event={"ID":"03d489c4-e94c-45ed-82f8-3b0e2b389e9d","Type":"ContainerStarted","Data":"04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006"}
Sep 08 14:36:36.019676 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:36.019524    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" event={"ID":"03d489c4-e94c-45ed-82f8-3b0e2b389e9d","Type":"ContainerStarted","Data":"209e0b531843d0e5806b7052cc705160328240c2f71eb96e49d6259e2946bf26"}
Sep 08 14:36:36.040587 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:36.040530    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" podStartSLOduration=2.040512224 podStartE2EDuration="2.040512224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:36:36.03806738 +0000 UTC m=+1211.268998577" watchObservedRunningTime="2026-09-08 14:36:36.040512224 +0000 UTC m=+1211.271443420"
Sep 08 14:36:36.921252 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:36.921188    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:36.926706 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:36.926680    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:37.025027 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:37.024994    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:37.025988 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:37.025969    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:37.936336 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:37.936302    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:38.036217 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:38.036183    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:38.480657 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:38.480625    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:38.480843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:38.480760    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:38.493382 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:38.493350    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:39.046811 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:39.046782    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:36:39.127272 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:39.127195    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:39.227514 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:39.227486    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:39.327149 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:39.327066    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:39.628615 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:39.628534    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:39.728764 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:39.728735    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:46.153934 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:46.153895    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:36:51.474225 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.474168    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"]
Sep 08 14:36:51.477500 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.477468    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:51.481284 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.481254    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"]
Sep 08 14:36:51.481421 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.481371    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.484047 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.484018    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-017dbd\"/\"kube-root-ca.crt\""
Sep 08 14:36:51.484256 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.484190    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-017dbd\"/\"openshift-service-ca.crt\""
Sep 08 14:36:51.485441 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.485419    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-017dbd\"/\"default-dockercfg-frkbm\""
Sep 08 14:36:51.485559 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.485511    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-017dbd\"/\"auth-test-model-017dbd-kserve-self-signed-certs\""
Sep 08 14:36:51.596929 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.596889    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tls-certs\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.597156 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.597015    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-dshm\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.597156 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.597076    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-home\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.597156 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.597142    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wccn\" (UniqueName: \"kubernetes.io/projected/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-kube-api-access-9wccn\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.597327 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.597167    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-model-cache\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.597327 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.597186    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tmp-dir\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.698196 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.698157    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wccn\" (UniqueName: \"kubernetes.io/projected/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-kube-api-access-9wccn\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.698196 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.698199    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-model-cache\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.698461 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.698238    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tmp-dir\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.698461 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.698298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tls-certs\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.698461 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.698359    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-dshm\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.698461 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.698378    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-home\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.698753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.698725    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-model-cache\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.698753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.698741    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tmp-dir\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.698862 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.698792    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-home\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.700722 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.700697    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-dshm\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.700987 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.700965    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tls-certs\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.708849 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.708824    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wccn\" (UniqueName: \"kubernetes.io/projected/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-kube-api-access-9wccn\") pod \"auth-test-model-017dbd-kserve-7555786c95-4krn4\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:51.794333 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:51.794291    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:36:52.090259 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:52.090228    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"]
Sep 08 14:36:52.092394 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:36:52.092368    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2b6d9bf_9ff3_4c90_aae8_47edc8655e55.slice/crio-a20ac154c3776af9bf51e969973326cd7c6b2b079472cac0e0979e06d67a6f89 WatchSource:0}: Error finding container a20ac154c3776af9bf51e969973326cd7c6b2b079472cac0e0979e06d67a6f89: Status 404 returned error can't find the container with id a20ac154c3776af9bf51e969973326cd7c6b2b079472cac0e0979e06d67a6f89
Sep 08 14:36:52.112786 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:52.112751    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4" event={"ID":"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55","Type":"ContainerStarted","Data":"a20ac154c3776af9bf51e969973326cd7c6b2b079472cac0e0979e06d67a6f89"}
Sep 08 14:36:53.120101 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:53.120062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4" event={"ID":"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55","Type":"ContainerStarted","Data":"f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c"}
Sep 08 14:36:53.139042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:53.138983    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4" podStartSLOduration=2.138966698 podStartE2EDuration="2.138966698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:36:53.135774935 +0000 UTC m=+1228.366706135" watchObservedRunningTime="2026-09-08 14:36:53.138966698 +0000 UTC m=+1228.369897927"
Sep 08 14:36:56.137857 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:56.137820    2558 generic.go:356] "Generic (PLEG): container finished" podID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerID="04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006" exitCode=0
Sep 08 14:36:56.138398 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:56.137893    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" event={"ID":"03d489c4-e94c-45ed-82f8-3b0e2b389e9d","Type":"ContainerDied","Data":"04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006"}
Sep 08 14:36:56.138497 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:56.138479    2558 scope.go:117] "RemoveContainer" containerID="04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006"
Sep 08 14:36:56.139767 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:56.139738    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:36:56.139847 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:56.139805    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:36:56.139887 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:56.139851    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:36:57.145715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:57.145671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" event={"ID":"03d489c4-e94c-45ed-82f8-3b0e2b389e9d","Type":"ContainerStarted","Data":"89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca"}
Sep 08 14:36:57.665537 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:57.665478    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:57.921021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:57.920929    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:36:57.921199 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:57.921132    2558 patch_prober.go:28] interesting pod/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 08 14:36:57.921308 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:57.921186    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 08 14:36:58.863068 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:58.863006    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:36:58.921064 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:58.921026    2558 patch_prober.go:28] interesting pod/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 08 14:36:58.921253 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:58.921090    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 08 14:36:59.921291 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:59.921248    2558 patch_prober.go:28] interesting pod/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 08 14:36:59.921661 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:36:59.921298    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 08 14:37:00.921867 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:00.921831    2558 patch_prober.go:28] interesting pod/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 08 14:37:00.922310 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:00.921888    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 08 14:37:01.117195 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:01.117160    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:01.795047 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:01.795004    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:37:01.795269 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:01.795097    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:37:01.807701 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:01.807672    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:37:01.921938 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:01.921900    2558 patch_prober.go:28] interesting pod/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 08 14:37:01.922353 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:01.921957    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 08 14:37:02.205026 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:02.204941    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:37:02.925719 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:02.925687    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:37:02.926088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:02.926020    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:37:02.926621 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:02.926601    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:37:03.323172 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:03.323137    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:12.744798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:12.744752    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:19.459721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:19.459639    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:37:38.549276 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:38.549233    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:37:53.228927 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:53.228896    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:53.300938 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:53.300907    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:53.363338 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:53.363306    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:53.439848 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:53.439818    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:53.497068 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:53.497015    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:56.951339 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:56.951306    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:37:57.060168 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:37:57.060118    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:02.082309 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:02.082270    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:04.095611 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:04.095575    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:04.150708 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:04.150676    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:13.855742 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:13.855703    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:38:30.152824 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:30.152789    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:38:38.281857 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:38.281819    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:39.469108 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:39.469066    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:44.342943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:44.342909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:44.398895 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:44.398855    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:45.186473 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:45.186443    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:45.250323 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:45.250226    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:45.469009 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:45.468929    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"]
Sep 08 14:38:45.469418 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:45.469127    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy" containerID="cri-o://89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca" gracePeriod=30
Sep 08 14:38:45.527256 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:45.527215    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:46.348464 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:46.348428    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:38:46.809584 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:46.809544    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:46.909611 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:46.909577    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:47.013008 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:47.012973    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:47.689304 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:47.689172    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:47.789380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:47.789347    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:47.890951 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:47.890915    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:47.928113 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:47.928084    2558 patch_prober.go:28] interesting pod/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 14:38:47.928262 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:47.928137    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 14:38:49.260909 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:49.260870    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:50.725754 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.725730    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:38:50.749512 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.749486    2558 generic.go:356] "Generic (PLEG): container finished" podID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerID="89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca" exitCode=0
Sep 08 14:38:50.749512 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.749520    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" event={"ID":"03d489c4-e94c-45ed-82f8-3b0e2b389e9d","Type":"ContainerDied","Data":"89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca"}
Sep 08 14:38:50.749742 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.749552    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"
Sep 08 14:38:50.749742 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.749571    2558 scope.go:117] "RemoveContainer" containerID="89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca"
Sep 08 14:38:50.749742 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.749560    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq" event={"ID":"03d489c4-e94c-45ed-82f8-3b0e2b389e9d","Type":"ContainerDied","Data":"209e0b531843d0e5806b7052cc705160328240c2f71eb96e49d6259e2946bf26"}
Sep 08 14:38:50.759493 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.759477    2558 scope.go:117] "RemoveContainer" containerID="04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006"
Sep 08 14:38:50.769080 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.769060    2558 scope.go:117] "RemoveContainer" containerID="89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca"
Sep 08 14:38:50.769403 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:38:50.769377    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca\": container with ID starting with 89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca not found: ID does not exist" containerID="89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca"
Sep 08 14:38:50.769500 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.769414    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca"} err="failed to get container status \"89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca\": rpc error: code = NotFound desc = could not find container \"89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca\": container with ID starting with 89400c34b1ebda8891d31d0cc21efae9580e2a5cc7bfd47dc96cba747e0626ca not found: ID does not exist"
Sep 08 14:38:50.769500 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.769438    2558 scope.go:117] "RemoveContainer" containerID="04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006"
Sep 08 14:38:50.769749 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:38:50.769731    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006\": container with ID starting with 04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006 not found: ID does not exist" containerID="04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006"
Sep 08 14:38:50.769813 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.769755    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006"} err="failed to get container status \"04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006\": rpc error: code = NotFound desc = could not find container \"04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006\": container with ID starting with 04596e639483f34f21283484883705e9115515caf8c85eeb2be902d86066b006 not found: ID does not exist"
Sep 08 14:38:50.802786 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.802751    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-socket\") pod \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") "
Sep 08 14:38:50.802786 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.802791    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-envoy\") pod \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") "
Sep 08 14:38:50.803006 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.802850    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-credential-socket\") pod \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") "
Sep 08 14:38:50.803006 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.802884    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-token\") pod \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") "
Sep 08 14:38:50.803006 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.802910    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istiod-ca-cert\") pod \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") "
Sep 08 14:38:50.803006 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.802946    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-podinfo\") pod \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") "
Sep 08 14:38:50.803251 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803046    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnddb\" (UniqueName: \"kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-kube-api-access-cnddb\") pod \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") "
Sep 08 14:38:50.803251 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803060    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "03d489c4-e94c-45ed-82f8-3b0e2b389e9d" (UID: "03d489c4-e94c-45ed-82f8-3b0e2b389e9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:38:50.803251 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803076    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-certs\") pod \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") "
Sep 08 14:38:50.803251 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803175    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-data\") pod \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\" (UID: \"03d489c4-e94c-45ed-82f8-3b0e2b389e9d\") "
Sep 08 14:38:50.803251 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803231    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "03d489c4-e94c-45ed-82f8-3b0e2b389e9d" (UID: "03d489c4-e94c-45ed-82f8-3b0e2b389e9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:38:50.803491 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803342    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "03d489c4-e94c-45ed-82f8-3b0e2b389e9d" (UID: "03d489c4-e94c-45ed-82f8-3b0e2b389e9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:38:50.803491 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803434    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "03d489c4-e94c-45ed-82f8-3b0e2b389e9d" (UID: "03d489c4-e94c-45ed-82f8-3b0e2b389e9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:38:50.803656 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803635    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-data" (OuterVolumeSpecName: "istio-data") pod "03d489c4-e94c-45ed-82f8-3b0e2b389e9d" (UID: "03d489c4-e94c-45ed-82f8-3b0e2b389e9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:38:50.803753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803732    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-credential-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:38:50.803832 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803757    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istiod-ca-cert\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:38:50.803832 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803772    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:38:50.803832 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.803790    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-workload-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:38:50.805275 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.805197    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "03d489c4-e94c-45ed-82f8-3b0e2b389e9d" (UID: "03d489c4-e94c-45ed-82f8-3b0e2b389e9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:38:50.805368 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.805282    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-token" (OuterVolumeSpecName: "istio-token") pod "03d489c4-e94c-45ed-82f8-3b0e2b389e9d" (UID: "03d489c4-e94c-45ed-82f8-3b0e2b389e9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:38:50.805368 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.805310    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "03d489c4-e94c-45ed-82f8-3b0e2b389e9d" (UID: "03d489c4-e94c-45ed-82f8-3b0e2b389e9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:38:50.805368 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.805309    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-kube-api-access-cnddb" (OuterVolumeSpecName: "kube-api-access-cnddb") pod "03d489c4-e94c-45ed-82f8-3b0e2b389e9d" (UID: "03d489c4-e94c-45ed-82f8-3b0e2b389e9d"). InnerVolumeSpecName "kube-api-access-cnddb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:38:50.904818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.904771    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-data\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:38:50.904818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.904808    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-envoy\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:38:50.904818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.904819    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-token\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:38:50.904818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.904829    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-istio-podinfo\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:38:50.905088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:50.904842    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnddb\" (UniqueName: \"kubernetes.io/projected/03d489c4-e94c-45ed-82f8-3b0e2b389e9d-kube-api-access-cnddb\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:38:51.074246 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:51.074216    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"]
Sep 08 14:38:51.078811 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:51.078786    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-afb814fe-openshift-default-c7b76db44-bpbpq"]
Sep 08 14:38:51.287896 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:51.287862    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" path="/var/lib/kubelet/pods/03d489c4-e94c-45ed-82f8-3b0e2b389e9d/volumes"
Sep 08 14:38:55.222025 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:55.221987    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:55.321743 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:55.321715    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:55.925975 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:38:55.925938    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:00.969920 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:00.969882    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"]
Sep 08 14:39:00.995801 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:00.995773    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:01.146996 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:01.146961    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:39:01.875899 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:01.875862    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"]
Sep 08 14:39:01.911387 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:01.911338    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:02.488871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:02.488837    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:02.958764 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:02.958678    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:03.059336 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:03.059303    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:05.337158 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.337124    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"]
Sep 08 14:39:05.337586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.337551    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy"
Sep 08 14:39:05.337586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.337564    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy"
Sep 08 14:39:05.337586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.337585    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy"
Sep 08 14:39:05.337757 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.337593    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy"
Sep 08 14:39:05.337757 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.337679    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy"
Sep 08 14:39:05.337757 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.337692    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="03d489c4-e94c-45ed-82f8-3b0e2b389e9d" containerName="istio-proxy"
Sep 08 14:39:05.340915 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.340891    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.343351 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.343325    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-780944\"/\"sub-test-model-780944-kserve-self-signed-certs\""
Sep 08 14:39:05.352798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.352775    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"]
Sep 08 14:39:05.366717 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.366684    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:05.439241 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.439172    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed71ede-81e9-4003-8140-6f8bba0941e4-tls-certs\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.439396 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.439293    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-tmp-dir\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.439396 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.439335    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-dshm\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.439473 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.439412    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-model-cache\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.439473 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.439445    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-home\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.439539 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.439507    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4srnj\" (UniqueName: \"kubernetes.io/projected/0ed71ede-81e9-4003-8140-6f8bba0941e4-kube-api-access-4srnj\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.540038 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.540005    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed71ede-81e9-4003-8140-6f8bba0941e4-tls-certs\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.540250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.540078    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-tmp-dir\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.540250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.540109    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-dshm\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.540250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.540179    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-model-cache\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.540250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.540219    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-home\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.540475 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.540283    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4srnj\" (UniqueName: \"kubernetes.io/projected/0ed71ede-81e9-4003-8140-6f8bba0941e4-kube-api-access-4srnj\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.540566 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.540540    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-tmp-dir\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.540786 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.540602    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-model-cache\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.540884 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.540676    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-home\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.542432 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.542406    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-dshm\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.542763 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.542745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed71ede-81e9-4003-8140-6f8bba0941e4-tls-certs\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.548540 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.548517    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4srnj\" (UniqueName: \"kubernetes.io/projected/0ed71ede-81e9-4003-8140-6f8bba0941e4-kube-api-access-4srnj\") pod \"sub-test-model-780944-kserve-7459985779-5lclj\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") " pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.649734 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.649657    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:05.919345 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:05.919317    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"]
Sep 08 14:39:05.920802 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:39:05.920770    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ed71ede_81e9_4003_8140_6f8bba0941e4.slice/crio-eacb1b847719ccfb188ee9cf95e1ec1bc088af6d8f98477cf56e968da0c068a1 WatchSource:0}: Error finding container eacb1b847719ccfb188ee9cf95e1ec1bc088af6d8f98477cf56e968da0c068a1: Status 404 returned error can't find the container with id eacb1b847719ccfb188ee9cf95e1ec1bc088af6d8f98477cf56e968da0c068a1
Sep 08 14:39:06.843599 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:06.843556    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj" event={"ID":"0ed71ede-81e9-4003-8140-6f8bba0941e4","Type":"ContainerStarted","Data":"0de4d8069eacbabd2024a88b7373691053fc259d0cd9df762f54dfb73fae2d45"}
Sep 08 14:39:06.843987 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:06.843611    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj" event={"ID":"0ed71ede-81e9-4003-8140-6f8bba0941e4","Type":"ContainerStarted","Data":"eacb1b847719ccfb188ee9cf95e1ec1bc088af6d8f98477cf56e968da0c068a1"}
Sep 08 14:39:06.864987 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:06.864936    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj" podStartSLOduration=1.864921491 podStartE2EDuration="1.864921491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:39:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:39:06.863692733 +0000 UTC m=+1362.094623931" watchObservedRunningTime="2026-09-08 14:39:06.864921491 +0000 UTC m=+1362.095852715"
Sep 08 14:39:13.231019 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:13.230979    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:13.484524 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:13.484498    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:15.649862 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:15.649825    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:15.650460 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:15.649882    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:15.662881 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:15.662849    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:15.906401 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:15.906316    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:39:15.989652 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:15.989612    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns" podUID="377f1853-1121-47c8-ac3b-9c97f1ec8463" containerName="main" containerID="cri-o://f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3" gracePeriod=45
Sep 08 14:39:16.244370 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.244338    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:39:16.343488 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.343455    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/377f1853-1121-47c8-ac3b-9c97f1ec8463-tls-certs\") pod \"377f1853-1121-47c8-ac3b-9c97f1ec8463\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") "
Sep 08 14:39:16.343653 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.343500    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-model-cache\") pod \"377f1853-1121-47c8-ac3b-9c97f1ec8463\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") "
Sep 08 14:39:16.343653 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.343538    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-home\") pod \"377f1853-1121-47c8-ac3b-9c97f1ec8463\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") "
Sep 08 14:39:16.343653 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.343586    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs8bf\" (UniqueName: \"kubernetes.io/projected/377f1853-1121-47c8-ac3b-9c97f1ec8463-kube-api-access-qs8bf\") pod \"377f1853-1121-47c8-ac3b-9c97f1ec8463\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") "
Sep 08 14:39:16.343653 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.343646    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-tmp-dir\") pod \"377f1853-1121-47c8-ac3b-9c97f1ec8463\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") "
Sep 08 14:39:16.343846 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.343671    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-dshm\") pod \"377f1853-1121-47c8-ac3b-9c97f1ec8463\" (UID: \"377f1853-1121-47c8-ac3b-9c97f1ec8463\") "
Sep 08 14:39:16.343979 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.343843    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-model-cache" (OuterVolumeSpecName: "model-cache") pod "377f1853-1121-47c8-ac3b-9c97f1ec8463" (UID: "377f1853-1121-47c8-ac3b-9c97f1ec8463"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:39:16.343979 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.343856    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-home" (OuterVolumeSpecName: "home") pod "377f1853-1121-47c8-ac3b-9c97f1ec8463" (UID: "377f1853-1121-47c8-ac3b-9c97f1ec8463"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:39:16.344082 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.343980    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "377f1853-1121-47c8-ac3b-9c97f1ec8463" (UID: "377f1853-1121-47c8-ac3b-9c97f1ec8463"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:39:16.344082 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.344045    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-home\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:16.344082 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.344058    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-model-cache\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:16.345820 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.345796    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/377f1853-1121-47c8-ac3b-9c97f1ec8463-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "377f1853-1121-47c8-ac3b-9c97f1ec8463" (UID: "377f1853-1121-47c8-ac3b-9c97f1ec8463"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:39:16.345923 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.345815    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/377f1853-1121-47c8-ac3b-9c97f1ec8463-kube-api-access-qs8bf" (OuterVolumeSpecName: "kube-api-access-qs8bf") pod "377f1853-1121-47c8-ac3b-9c97f1ec8463" (UID: "377f1853-1121-47c8-ac3b-9c97f1ec8463"). InnerVolumeSpecName "kube-api-access-qs8bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:39:16.345923 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.345855    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-dshm" (OuterVolumeSpecName: "dshm") pod "377f1853-1121-47c8-ac3b-9c97f1ec8463" (UID: "377f1853-1121-47c8-ac3b-9c97f1ec8463"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:39:16.445300 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.445257    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs8bf\" (UniqueName: \"kubernetes.io/projected/377f1853-1121-47c8-ac3b-9c97f1ec8463-kube-api-access-qs8bf\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:16.445300 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.445293    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-tmp-dir\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:16.445300 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.445308    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/377f1853-1121-47c8-ac3b-9c97f1ec8463-dshm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:16.445538 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.445320    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/377f1853-1121-47c8-ac3b-9c97f1ec8463-tls-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:16.850060 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.850027    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:39:16.894769 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.894736    2558 generic.go:356] "Generic (PLEG): container finished" podID="377f1853-1121-47c8-ac3b-9c97f1ec8463" containerID="f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3" exitCode=0
Sep 08 14:39:16.894921 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.894777    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"
Sep 08 14:39:16.894921 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.894822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns" event={"ID":"377f1853-1121-47c8-ac3b-9c97f1ec8463","Type":"ContainerDied","Data":"f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3"}
Sep 08 14:39:16.894921 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.894869    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns" event={"ID":"377f1853-1121-47c8-ac3b-9c97f1ec8463","Type":"ContainerDied","Data":"5f6e810990d94b14891bca80eb37ac0111544c014f9bed4346c74c20e9523725"}
Sep 08 14:39:16.894921 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.894890    2558 scope.go:117] "RemoveContainer" containerID="f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3"
Sep 08 14:39:16.896621 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.896595    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4" podUID="f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" containerName="main" containerID="cri-o://f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c" gracePeriod=45
Sep 08 14:39:16.905189 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.905171    2558 scope.go:117] "RemoveContainer" containerID="f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3"
Sep 08 14:39:16.905499 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:39:16.905480    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3\": container with ID starting with f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3 not found: ID does not exist" containerID="f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3"
Sep 08 14:39:16.905567 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.905507    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3"} err="failed to get container status \"f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3\": rpc error: code = NotFound desc = could not find container \"f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3\": container with ID starting with f18a0154cf4a5f1f8a01d7b121cf04204929d561167cfbd6f2bbd8f67c93e7a3 not found: ID does not exist"
Sep 08 14:39:16.917834 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.917802    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"]
Sep 08 14:39:16.919727 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:16.919708    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/auth-test-model-780944-kserve-5c8997794c-7ttns"]
Sep 08 14:39:17.151593 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.151566    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:39:17.251910 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.251871    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-model-cache\") pod \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") "
Sep 08 14:39:17.252111 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.251966    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-home\") pod \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") "
Sep 08 14:39:17.252111 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.251991    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tmp-dir\") pod \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") "
Sep 08 14:39:17.252111 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.252034    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-dshm\") pod \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") "
Sep 08 14:39:17.252111 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.252084    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tls-certs\") pod \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") "
Sep 08 14:39:17.252376 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.252121    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wccn\" (UniqueName: \"kubernetes.io/projected/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-kube-api-access-9wccn\") pod \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\" (UID: \"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55\") "
Sep 08 14:39:17.252376 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.252184    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-model-cache" (OuterVolumeSpecName: "model-cache") pod "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" (UID: "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:39:17.252376 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.252238    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" (UID: "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:39:17.252376 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.252293    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-home" (OuterVolumeSpecName: "home") pod "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" (UID: "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:39:17.252595 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.252523    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-model-cache\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:17.252595 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.252542    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-home\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:17.252595 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.252554    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tmp-dir\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:17.254528 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.254500    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" (UID: "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:39:17.254528 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.254513    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-kube-api-access-9wccn" (OuterVolumeSpecName: "kube-api-access-9wccn") pod "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" (UID: "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55"). InnerVolumeSpecName "kube-api-access-9wccn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:39:17.254692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.254631    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-dshm" (OuterVolumeSpecName: "dshm") pod "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" (UID: "f2b6d9bf-9ff3-4c90-aae8-47edc8655e55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:39:17.287471 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.287432    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="377f1853-1121-47c8-ac3b-9c97f1ec8463" path="/var/lib/kubelet/pods/377f1853-1121-47c8-ac3b-9c97f1ec8463/volumes"
Sep 08 14:39:17.353185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.353147    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-tls-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:17.353185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.353182    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wccn\" (UniqueName: \"kubernetes.io/projected/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-kube-api-access-9wccn\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:17.353394 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.353197    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55-dshm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:39:17.901648 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.901608    2558 generic.go:356] "Generic (PLEG): container finished" podID="f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" containerID="f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c" exitCode=0
Sep 08 14:39:17.901648 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.901653    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"
Sep 08 14:39:17.902102 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.901690    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4" event={"ID":"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55","Type":"ContainerDied","Data":"f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c"}
Sep 08 14:39:17.902102 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.901736    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4" event={"ID":"f2b6d9bf-9ff3-4c90-aae8-47edc8655e55","Type":"ContainerDied","Data":"a20ac154c3776af9bf51e969973326cd7c6b2b079472cac0e0979e06d67a6f89"}
Sep 08 14:39:17.902102 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.901754    2558 scope.go:117] "RemoveContainer" containerID="f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c"
Sep 08 14:39:17.910335 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.910311    2558 scope.go:117] "RemoveContainer" containerID="f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c"
Sep 08 14:39:17.910616 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:39:17.910597    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c\": container with ID starting with f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c not found: ID does not exist" containerID="f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c"
Sep 08 14:39:17.910665 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.910625    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c"} err="failed to get container status \"f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c\": rpc error: code = NotFound desc = could not find container \"f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c\": container with ID starting with f91bd63d218e7721e6b728a6968d92914e67e439394a28a3f162b4c8f04c876c not found: ID does not exist"
Sep 08 14:39:17.919483 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.919455    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"]
Sep 08 14:39:17.923322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:17.923298    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/auth-test-model-017dbd-kserve-7555786c95-4krn4"]
Sep 08 14:39:19.287639 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:19.287603    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" path="/var/lib/kubelet/pods/f2b6d9bf-9ff3-4c90-aae8-47edc8655e55/volumes"
Sep 08 14:39:28.461602 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.461566    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"]
Sep 08 14:39:28.462020 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.462006    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" containerName="main"
Sep 08 14:39:28.462020 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.462022    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" containerName="main"
Sep 08 14:39:28.462096 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.462045    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="377f1853-1121-47c8-ac3b-9c97f1ec8463" containerName="main"
Sep 08 14:39:28.462096 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.462051    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="377f1853-1121-47c8-ac3b-9c97f1ec8463" containerName="main"
Sep 08 14:39:28.462159 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.462129    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2b6d9bf-9ff3-4c90-aae8-47edc8655e55" containerName="main"
Sep 08 14:39:28.462159 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.462141    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="377f1853-1121-47c8-ac3b-9c97f1ec8463" containerName="main"
Sep 08 14:39:28.466616 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.466591    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.470457 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.470436    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-017dbd\"/\"default-dockercfg-frkbm\""
Sep 08 14:39:28.470577 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.470457    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-017dbd\"/\"openshift-service-ca.crt\""
Sep 08 14:39:28.470577 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.470436    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-017dbd\"/\"kube-root-ca.crt\""
Sep 08 14:39:28.470577 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.470436    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-017dbd\"/\"sub-test-model-017dbd-kserve-self-signed-certs\""
Sep 08 14:39:28.474395 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.474373    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"]
Sep 08 14:39:28.488713 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.488684    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:28.567038 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.567005    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56e3c0f6-4f78-4100-a718-7285998c3354-tls-certs\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.567189 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.567052    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-home\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.567189 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.567109    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-tmp-dir\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.567189 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.567178    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-dshm\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.567345 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.567235    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-model-cache\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.567345 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.567254    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp9vf\" (UniqueName: \"kubernetes.io/projected/56e3c0f6-4f78-4100-a718-7285998c3354-kube-api-access-tp9vf\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.668764 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.668724    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-home\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.668936 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.668769    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-tmp-dir\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.668936 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.668815    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-dshm\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.668936 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.668857    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-model-cache\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.668936 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.668884    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp9vf\" (UniqueName: \"kubernetes.io/projected/56e3c0f6-4f78-4100-a718-7285998c3354-kube-api-access-tp9vf\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.669129 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.668978    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56e3c0f6-4f78-4100-a718-7285998c3354-tls-certs\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.669185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.669166    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-home\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.669286 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.669249    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-tmp-dir\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.669344 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.669303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-model-cache\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.671190 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.671165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-dshm\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.671373 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.671354    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56e3c0f6-4f78-4100-a718-7285998c3354-tls-certs\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.677757 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.677735    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp9vf\" (UniqueName: \"kubernetes.io/projected/56e3c0f6-4f78-4100-a718-7285998c3354-kube-api-access-tp9vf\") pod \"sub-test-model-017dbd-kserve-55bb677c6c-htrmm\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") " pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:28.776309 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:28.776264    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:29.046296 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:29.046267    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"]
Sep 08 14:39:29.047987 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:39:29.047959    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56e3c0f6_4f78_4100_a718_7285998c3354.slice/crio-f07d9195ae390885642871cdd7344e1716c307d7f2079642c316342e66d50b72 WatchSource:0}: Error finding container f07d9195ae390885642871cdd7344e1716c307d7f2079642c316342e66d50b72: Status 404 returned error can't find the container with id f07d9195ae390885642871cdd7344e1716c307d7f2079642c316342e66d50b72
Sep 08 14:39:29.962866 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:29.962828    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm" event={"ID":"56e3c0f6-4f78-4100-a718-7285998c3354","Type":"ContainerStarted","Data":"2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e"}
Sep 08 14:39:29.962866 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:29.962870    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm" event={"ID":"56e3c0f6-4f78-4100-a718-7285998c3354","Type":"ContainerStarted","Data":"f07d9195ae390885642871cdd7344e1716c307d7f2079642c316342e66d50b72"}
Sep 08 14:39:29.981970 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:29.981921    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm" podStartSLOduration=1.981906027 podStartE2EDuration="1.981906027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:39:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:39:29.981054698 +0000 UTC m=+1385.211985897" watchObservedRunningTime="2026-09-08 14:39:29.981906027 +0000 UTC m=+1385.212837223"
Sep 08 14:39:32.857996 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:32.857958    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:39:34.457285 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:34.457251    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:38.776692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:38.776658    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:38.777095 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:38.776978    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:38.789641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:38.789609    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:39.018351 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:39.018321    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:39:40.545464 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:40.545432    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:40.609585 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:40.609556    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:40.916862 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:40.916758    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:41.177280 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.177178    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"]
Sep 08 14:39:41.180948 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.180927    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.183738 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.183714    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-0fd47b-openshift-default-dockercfg-67fsq\""
Sep 08 14:39:41.192973 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.192952    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"]
Sep 08 14:39:41.202095 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.202068    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:41.299624 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.299584    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-podinfo\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.299787 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.299633    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-socket\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.299787 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.299699    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-token\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.299919 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.299786    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-credential-socket\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.299919 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.299827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-envoy\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.300005 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.299926    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-data\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.300005 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.299950    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jlfd\" (UniqueName: \"kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-kube-api-access-4jlfd\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.300005 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.299977    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-certs\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.300114 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.300002    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istiod-ca-cert\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.377780 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.377746    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:41.400972 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.400938    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-socket\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.400992    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-token\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401038    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-credential-socket\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-envoy\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401156    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-data\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401187    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jlfd\" (UniqueName: \"kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-kube-api-access-4jlfd\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401256    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-certs\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401282    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istiod-ca-cert\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401517 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401339    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-podinfo\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401517 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401455    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-socket\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401517 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401498    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-credential-socket\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401681 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401618    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-data\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401681 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401653    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-certs\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.401991 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.401971    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istiod-ca-cert\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.403527 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.403505    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-envoy\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.403815 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.403794    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-podinfo\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.410556 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.410533    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jlfd\" (UniqueName: \"kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-kube-api-access-4jlfd\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.410861 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.410841    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-token\") pod \"e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") " pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.493283 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.493257    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:41.768655 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.768623    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"]
Sep 08 14:39:41.770068 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:39:41.770037    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f6d56b_bfb2_42b6_93c5_065689ffcc7b.slice/crio-ce2add4f38248324548d40ad77e5d163c6c65688d34a9fb4e2fe37fea851db36 WatchSource:0}: Error finding container ce2add4f38248324548d40ad77e5d163c6c65688d34a9fb4e2fe37fea851db36: Status 404 returned error can't find the container with id ce2add4f38248324548d40ad77e5d163c6c65688d34a9fb4e2fe37fea851db36
Sep 08 14:39:41.772705 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.772659    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:39:41.772812 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.772791    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:39:41.772855 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:41.772842    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:39:42.025643 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:42.025561    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" event={"ID":"06f6d56b-bfb2-42b6-93c5-065689ffcc7b","Type":"ContainerStarted","Data":"9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2"}
Sep 08 14:39:42.025643 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:42.025612    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" event={"ID":"06f6d56b-bfb2-42b6-93c5-065689ffcc7b","Type":"ContainerStarted","Data":"ce2add4f38248324548d40ad77e5d163c6c65688d34a9fb4e2fe37fea851db36"}
Sep 08 14:39:42.046747 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:42.046693    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" podStartSLOduration=1.046679579 podStartE2EDuration="1.046679579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:39:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:39:42.044673843 +0000 UTC m=+1397.275605041" watchObservedRunningTime="2026-09-08 14:39:42.046679579 +0000 UTC m=+1397.277610843"
Sep 08 14:39:42.494101 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:42.494067    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:42.495441 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:42.495414    2558 patch_prober.go:28] interesting pod/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 08 14:39:42.495558 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:42.495467    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 08 14:39:43.498452 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:43.498420    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:44.021574 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:44.021538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:44.036924 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:44.036894    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:44.037972 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:44.037945    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:39:44.116425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:44.116396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:45.174385 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:45.174350    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:45.229262 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:45.229224    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:45.330012 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:45.329979    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:45.678231 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:45.678187    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:45.777420 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:45.777387    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:52.462779 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:39:52.462740    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:40:03.140712 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:03.140676    2558 generic.go:356] "Generic (PLEG): container finished" podID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerID="9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2" exitCode=0
Sep 08 14:40:03.141135 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:03.140746    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" event={"ID":"06f6d56b-bfb2-42b6-93c5-065689ffcc7b","Type":"ContainerDied","Data":"9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2"}
Sep 08 14:40:03.141191 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:03.141176    2558 scope.go:117] "RemoveContainer" containerID="9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2"
Sep 08 14:40:03.142388 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:03.142358    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:40:03.142492 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:03.142436    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:40:03.142492 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:03.142467    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:40:04.149611 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:04.149566    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" event={"ID":"06f6d56b-bfb2-42b6-93c5-065689ffcc7b","Type":"ContainerStarted","Data":"4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98"}
Sep 08 14:40:04.493713 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:04.493673    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:40:04.493963 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:04.493938    2558 patch_prober.go:28] interesting pod/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 08 14:40:04.494059 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:04.494010    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 08 14:40:05.494425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:05.494388    2558 patch_prober.go:28] interesting pod/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 08 14:40:05.494816 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:05.494447    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 08 14:40:06.494435 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:06.494398    2558 patch_prober.go:28] interesting pod/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 08 14:40:06.494797 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:06.494467    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 08 14:40:07.494028 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:07.493991    2558 patch_prober.go:28] interesting pod/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 08 14:40:07.494242 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:07.494050    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 08 14:40:08.493822 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:08.493781    2558 patch_prober.go:28] interesting pod/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 08 14:40:08.494271 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:08.493838    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 08 14:40:09.040497 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:09.040460    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:09.575125 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:09.575076    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:09.587399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:09.587369    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:40:09.587930 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:09.587891    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:40:09.588574 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:09.588558    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:40:09.870269 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:09.870141    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:09.968791 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:09.968752    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:11.565585 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:11.565551    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:40:12.997534 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:12.997501    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:13.100049 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:13.100013    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:14.159263 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.159229    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:14.234482 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.234447    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:14.334330 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.334294    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:14.654694 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.654646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:14.757605 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.757575    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:14.766482 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.766454    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"]
Sep 08 14:40:14.769564 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.769549    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:40:14.772063 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.772036    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-41e512\""
Sep 08 14:40:14.772180 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.772085    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-41e512-dockercfg-m4qns\""
Sep 08 14:40:14.781727 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.781705    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"]
Sep 08 14:40:14.794609 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.794585    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"]
Sep 08 14:40:14.804798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.804758    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:40:14.807600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.807578    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"]
Sep 08 14:40:14.856322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.856287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl9ql\" (UniqueName: \"kubernetes.io/projected/2629b93f-6c64-4a5b-abbe-8374f3fc5947-kube-api-access-cl9ql\") pod \"payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l\" (UID: \"2629b93f-6c64-4a5b-abbe-8374f3fc5947\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:40:14.856461 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.856328    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2629b93f-6c64-4a5b-abbe-8374f3fc5947-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l\" (UID: \"2629b93f-6c64-4a5b-abbe-8374f3fc5947\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:40:14.856461 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.856449    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4gmq\" (UniqueName: \"kubernetes.io/projected/54a395cf-9c7b-4f1c-a12f-26104db18c99-kube-api-access-x4gmq\") pod \"payload-processing-e2e-api-b-41e512-85765778d5-x2725\" (UID: \"54a395cf-9c7b-4f1c-a12f-26104db18c99\") " pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:40:14.856586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.856495    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54a395cf-9c7b-4f1c-a12f-26104db18c99-plugins-config-volume\") pod \"payload-processing-e2e-api-b-41e512-85765778d5-x2725\" (UID: \"54a395cf-9c7b-4f1c-a12f-26104db18c99\") " pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:40:14.958039 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.957922    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl9ql\" (UniqueName: \"kubernetes.io/projected/2629b93f-6c64-4a5b-abbe-8374f3fc5947-kube-api-access-cl9ql\") pod \"payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l\" (UID: \"2629b93f-6c64-4a5b-abbe-8374f3fc5947\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:40:14.958039 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.957980    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2629b93f-6c64-4a5b-abbe-8374f3fc5947-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l\" (UID: \"2629b93f-6c64-4a5b-abbe-8374f3fc5947\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:40:14.958357 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.958067    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4gmq\" (UniqueName: \"kubernetes.io/projected/54a395cf-9c7b-4f1c-a12f-26104db18c99-kube-api-access-x4gmq\") pod \"payload-processing-e2e-api-b-41e512-85765778d5-x2725\" (UID: \"54a395cf-9c7b-4f1c-a12f-26104db18c99\") " pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:40:14.958357 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.958102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54a395cf-9c7b-4f1c-a12f-26104db18c99-plugins-config-volume\") pod \"payload-processing-e2e-api-b-41e512-85765778d5-x2725\" (UID: \"54a395cf-9c7b-4f1c-a12f-26104db18c99\") " pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:40:14.958772 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.958751    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2629b93f-6c64-4a5b-abbe-8374f3fc5947-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l\" (UID: \"2629b93f-6c64-4a5b-abbe-8374f3fc5947\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:40:14.958887 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.958869    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54a395cf-9c7b-4f1c-a12f-26104db18c99-plugins-config-volume\") pod \"payload-processing-e2e-api-b-41e512-85765778d5-x2725\" (UID: \"54a395cf-9c7b-4f1c-a12f-26104db18c99\") " pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:40:14.967482 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.967461    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4gmq\" (UniqueName: \"kubernetes.io/projected/54a395cf-9c7b-4f1c-a12f-26104db18c99-kube-api-access-x4gmq\") pod \"payload-processing-e2e-api-b-41e512-85765778d5-x2725\" (UID: \"54a395cf-9c7b-4f1c-a12f-26104db18c99\") " pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:40:14.967667 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:14.967645    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl9ql\" (UniqueName: \"kubernetes.io/projected/2629b93f-6c64-4a5b-abbe-8374f3fc5947-kube-api-access-cl9ql\") pod \"payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l\" (UID: \"2629b93f-6c64-4a5b-abbe-8374f3fc5947\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:40:15.080340 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:15.080301    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:40:15.115506 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:15.115474    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:40:15.350332 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:15.350301    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"]
Sep 08 14:40:15.352310 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:40:15.352280    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2629b93f_6c64_4a5b_abbe_8374f3fc5947.slice/crio-777132bfc71d315b265435c7d80c7523bf91cdfe2bc49f15b39ffc64cc625fe1 WatchSource:0}: Error finding container 777132bfc71d315b265435c7d80c7523bf91cdfe2bc49f15b39ffc64cc625fe1: Status 404 returned error can't find the container with id 777132bfc71d315b265435c7d80c7523bf91cdfe2bc49f15b39ffc64cc625fe1
Sep 08 14:40:15.395154 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:15.395129    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"]
Sep 08 14:40:15.396831 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:40:15.396801    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a395cf_9c7b_4f1c_a12f_26104db18c99.slice/crio-b1dfd8a404918b00dcfea318982a10adaa1d559b27a125692e77595565d1ffdd WatchSource:0}: Error finding container b1dfd8a404918b00dcfea318982a10adaa1d559b27a125692e77595565d1ffdd: Status 404 returned error can't find the container with id b1dfd8a404918b00dcfea318982a10adaa1d559b27a125692e77595565d1ffdd
Sep 08 14:40:16.220220 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:16.220167    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l" event={"ID":"2629b93f-6c64-4a5b-abbe-8374f3fc5947","Type":"ContainerStarted","Data":"714ac07a43df28a37be7b07e97ad6377f080303921d6b123cf527963357559eb"}
Sep 08 14:40:16.220430 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:16.220258    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l" event={"ID":"2629b93f-6c64-4a5b-abbe-8374f3fc5947","Type":"ContainerStarted","Data":"777132bfc71d315b265435c7d80c7523bf91cdfe2bc49f15b39ffc64cc625fe1"}
Sep 08 14:40:16.220430 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:16.220288    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:40:16.221895 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:16.221870    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725" event={"ID":"54a395cf-9c7b-4f1c-a12f-26104db18c99","Type":"ContainerStarted","Data":"a233435fc95530c5f59484b6284384fb0c38ba54789f4090ff0cee951dcc2fef"}
Sep 08 14:40:16.222039 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:16.221916    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725" event={"ID":"54a395cf-9c7b-4f1c-a12f-26104db18c99","Type":"ContainerStarted","Data":"b1dfd8a404918b00dcfea318982a10adaa1d559b27a125692e77595565d1ffdd"}
Sep 08 14:40:16.222039 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:16.221944    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:40:16.248083 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:16.248033    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l" podStartSLOduration=2.248019454 podStartE2EDuration="2.248019454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:40:16.245757654 +0000 UTC m=+1431.476688856" watchObservedRunningTime="2026-09-08 14:40:16.248019454 +0000 UTC m=+1431.478950714"
Sep 08 14:40:16.270284 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:16.270231    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725" podStartSLOduration=2.270198284 podStartE2EDuration="2.270198284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:40:16.267485323 +0000 UTC m=+1431.498416520" watchObservedRunningTime="2026-09-08 14:40:16.270198284 +0000 UTC m=+1431.501129481"
Sep 08 14:40:20.118177 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:20.118095    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"]
Sep 08 14:40:20.140712 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:20.140666    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:21.268464 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:21.268422    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"]
Sep 08 14:40:21.293470 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:21.293437    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:27.229411 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:27.229375    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:40:27.229992 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:27.229682    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:40:35.137988 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.137919    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj" podUID="0ed71ede-81e9-4003-8140-6f8bba0941e4" containerName="main" containerID="cri-o://0de4d8069eacbabd2024a88b7373691053fc259d0cd9df762f54dfb73fae2d45" gracePeriod=45
Sep 08 14:40:35.328766 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.328728    2558 generic.go:356] "Generic (PLEG): container finished" podID="0ed71ede-81e9-4003-8140-6f8bba0941e4" containerID="0de4d8069eacbabd2024a88b7373691053fc259d0cd9df762f54dfb73fae2d45" exitCode=0
Sep 08 14:40:35.328963 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.328790    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj" event={"ID":"0ed71ede-81e9-4003-8140-6f8bba0941e4","Type":"ContainerDied","Data":"0de4d8069eacbabd2024a88b7373691053fc259d0cd9df762f54dfb73fae2d45"}
Sep 08 14:40:35.395034 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.394967    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:40:35.464426 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.464392    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-dshm\") pod \"0ed71ede-81e9-4003-8140-6f8bba0941e4\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") "
Sep 08 14:40:35.464613 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.464459    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-home\") pod \"0ed71ede-81e9-4003-8140-6f8bba0941e4\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") "
Sep 08 14:40:35.464613 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.464511    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-tmp-dir\") pod \"0ed71ede-81e9-4003-8140-6f8bba0941e4\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") "
Sep 08 14:40:35.464613 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.464579    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-model-cache\") pod \"0ed71ede-81e9-4003-8140-6f8bba0941e4\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") "
Sep 08 14:40:35.464613 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.464601    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed71ede-81e9-4003-8140-6f8bba0941e4-tls-certs\") pod \"0ed71ede-81e9-4003-8140-6f8bba0941e4\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") "
Sep 08 14:40:35.464843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.464627    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4srnj\" (UniqueName: \"kubernetes.io/projected/0ed71ede-81e9-4003-8140-6f8bba0941e4-kube-api-access-4srnj\") pod \"0ed71ede-81e9-4003-8140-6f8bba0941e4\" (UID: \"0ed71ede-81e9-4003-8140-6f8bba0941e4\") "
Sep 08 14:40:35.464843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.464724    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-home" (OuterVolumeSpecName: "home") pod "0ed71ede-81e9-4003-8140-6f8bba0941e4" (UID: "0ed71ede-81e9-4003-8140-6f8bba0941e4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:40:35.464843 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.464736    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0ed71ede-81e9-4003-8140-6f8bba0941e4" (UID: "0ed71ede-81e9-4003-8140-6f8bba0941e4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:40:35.464959 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.464839    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-model-cache" (OuterVolumeSpecName: "model-cache") pod "0ed71ede-81e9-4003-8140-6f8bba0941e4" (UID: "0ed71ede-81e9-4003-8140-6f8bba0941e4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:40:35.465004 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.464985    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-model-cache\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:35.465069 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.465002    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-home\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:35.465069 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.465016    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-tmp-dir\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:35.466567 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.466533    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-dshm" (OuterVolumeSpecName: "dshm") pod "0ed71ede-81e9-4003-8140-6f8bba0941e4" (UID: "0ed71ede-81e9-4003-8140-6f8bba0941e4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:40:35.466567 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.466561    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ed71ede-81e9-4003-8140-6f8bba0941e4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0ed71ede-81e9-4003-8140-6f8bba0941e4" (UID: "0ed71ede-81e9-4003-8140-6f8bba0941e4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:40:35.466723 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.466632    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ed71ede-81e9-4003-8140-6f8bba0941e4-kube-api-access-4srnj" (OuterVolumeSpecName: "kube-api-access-4srnj") pod "0ed71ede-81e9-4003-8140-6f8bba0941e4" (UID: "0ed71ede-81e9-4003-8140-6f8bba0941e4"). InnerVolumeSpecName "kube-api-access-4srnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:40:35.566474 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.566441    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed71ede-81e9-4003-8140-6f8bba0941e4-tls-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:35.566634 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.566478    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4srnj\" (UniqueName: \"kubernetes.io/projected/0ed71ede-81e9-4003-8140-6f8bba0941e4-kube-api-access-4srnj\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:35.566634 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:35.566493    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ed71ede-81e9-4003-8140-6f8bba0941e4-dshm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:36.261735 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.261700    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:40:36.290579 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.290541    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm" podUID="56e3c0f6-4f78-4100-a718-7285998c3354" containerName="main" containerID="cri-o://2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e" gracePeriod=45
Sep 08 14:40:36.334881 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.334851    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"
Sep 08 14:40:36.335033 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.334853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj" event={"ID":"0ed71ede-81e9-4003-8140-6f8bba0941e4","Type":"ContainerDied","Data":"eacb1b847719ccfb188ee9cf95e1ec1bc088af6d8f98477cf56e968da0c068a1"}
Sep 08 14:40:36.335033 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.334985    2558 scope.go:117] "RemoveContainer" containerID="0de4d8069eacbabd2024a88b7373691053fc259d0cd9df762f54dfb73fae2d45"
Sep 08 14:40:36.370974 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.370931    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"]
Sep 08 14:40:36.375568 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.375545    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/sub-test-model-780944-kserve-7459985779-5lclj"]
Sep 08 14:40:36.546871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.546846    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:40:36.677891 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.677856    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56e3c0f6-4f78-4100-a718-7285998c3354-tls-certs\") pod \"56e3c0f6-4f78-4100-a718-7285998c3354\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") "
Sep 08 14:40:36.678069 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.677904    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-tmp-dir\") pod \"56e3c0f6-4f78-4100-a718-7285998c3354\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") "
Sep 08 14:40:36.678069 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.677979    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp9vf\" (UniqueName: \"kubernetes.io/projected/56e3c0f6-4f78-4100-a718-7285998c3354-kube-api-access-tp9vf\") pod \"56e3c0f6-4f78-4100-a718-7285998c3354\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") "
Sep 08 14:40:36.678069 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.678019    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-home\") pod \"56e3c0f6-4f78-4100-a718-7285998c3354\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") "
Sep 08 14:40:36.678069 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.678047    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-model-cache\") pod \"56e3c0f6-4f78-4100-a718-7285998c3354\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") "
Sep 08 14:40:36.678322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.678095    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-dshm\") pod \"56e3c0f6-4f78-4100-a718-7285998c3354\" (UID: \"56e3c0f6-4f78-4100-a718-7285998c3354\") "
Sep 08 14:40:36.678322 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.678217    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "56e3c0f6-4f78-4100-a718-7285998c3354" (UID: "56e3c0f6-4f78-4100-a718-7285998c3354"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:40:36.678429 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.678325    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-model-cache" (OuterVolumeSpecName: "model-cache") pod "56e3c0f6-4f78-4100-a718-7285998c3354" (UID: "56e3c0f6-4f78-4100-a718-7285998c3354"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:40:36.678429 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.678344    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-home" (OuterVolumeSpecName: "home") pod "56e3c0f6-4f78-4100-a718-7285998c3354" (UID: "56e3c0f6-4f78-4100-a718-7285998c3354"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:40:36.678542 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.678527    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-tmp-dir\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:36.678576 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.678545    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-home\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:36.678576 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.678554    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-model-cache\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:36.680080 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.680054    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56e3c0f6-4f78-4100-a718-7285998c3354-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "56e3c0f6-4f78-4100-a718-7285998c3354" (UID: "56e3c0f6-4f78-4100-a718-7285998c3354"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:40:36.680554 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.680535    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-dshm" (OuterVolumeSpecName: "dshm") pod "56e3c0f6-4f78-4100-a718-7285998c3354" (UID: "56e3c0f6-4f78-4100-a718-7285998c3354"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:40:36.680666 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.680554    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56e3c0f6-4f78-4100-a718-7285998c3354-kube-api-access-tp9vf" (OuterVolumeSpecName: "kube-api-access-tp9vf") pod "56e3c0f6-4f78-4100-a718-7285998c3354" (UID: "56e3c0f6-4f78-4100-a718-7285998c3354"). InnerVolumeSpecName "kube-api-access-tp9vf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:40:36.779153 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.779066    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56e3c0f6-4f78-4100-a718-7285998c3354-tls-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:36.779153 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.779101    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp9vf\" (UniqueName: \"kubernetes.io/projected/56e3c0f6-4f78-4100-a718-7285998c3354-kube-api-access-tp9vf\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:36.779153 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:36.779110    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56e3c0f6-4f78-4100-a718-7285998c3354-dshm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:40:37.289001 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:37.288965    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ed71ede-81e9-4003-8140-6f8bba0941e4" path="/var/lib/kubelet/pods/0ed71ede-81e9-4003-8140-6f8bba0941e4/volumes"
Sep 08 14:40:37.341348 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:37.341312    2558 generic.go:356] "Generic (PLEG): container finished" podID="56e3c0f6-4f78-4100-a718-7285998c3354" containerID="2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e" exitCode=0
Sep 08 14:40:37.341511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:37.341373    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"
Sep 08 14:40:37.341511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:37.341396    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm" event={"ID":"56e3c0f6-4f78-4100-a718-7285998c3354","Type":"ContainerDied","Data":"2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e"}
Sep 08 14:40:37.341511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:37.341439    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm" event={"ID":"56e3c0f6-4f78-4100-a718-7285998c3354","Type":"ContainerDied","Data":"f07d9195ae390885642871cdd7344e1716c307d7f2079642c316342e66d50b72"}
Sep 08 14:40:37.341511 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:37.341456    2558 scope.go:117] "RemoveContainer" containerID="2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e"
Sep 08 14:40:37.350710 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:37.350687    2558 scope.go:117] "RemoveContainer" containerID="2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e"
Sep 08 14:40:37.350971 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:40:37.350949    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e\": container with ID starting with 2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e not found: ID does not exist" containerID="2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e"
Sep 08 14:40:37.351054 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:37.350980    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e"} err="failed to get container status \"2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e\": rpc error: code = NotFound desc = could not find container \"2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e\": container with ID starting with 2f276231a65bc4ae9fd4a08e98656b29901597a17c984d43ea4ec7806725b05e not found: ID does not exist"
Sep 08 14:40:37.359657 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:37.359634    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"]
Sep 08 14:40:37.363909 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:37.363884    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-017dbd/sub-test-model-017dbd-kserve-55bb677c6c-htrmm"]
Sep 08 14:40:37.936340 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:37.936305    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:38.085155 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.085125    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"]
Sep 08 14:40:38.085580 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.085567    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56e3c0f6-4f78-4100-a718-7285998c3354" containerName="main"
Sep 08 14:40:38.085625 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.085582    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="56e3c0f6-4f78-4100-a718-7285998c3354" containerName="main"
Sep 08 14:40:38.085625 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.085604    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ed71ede-81e9-4003-8140-6f8bba0941e4" containerName="main"
Sep 08 14:40:38.085625 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.085610    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ed71ede-81e9-4003-8140-6f8bba0941e4" containerName="main"
Sep 08 14:40:38.085721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.085672    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ed71ede-81e9-4003-8140-6f8bba0941e4" containerName="main"
Sep 08 14:40:38.085721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.085684    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="56e3c0f6-4f78-4100-a718-7285998c3354" containerName="main"
Sep 08 14:40:38.088311 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.088292    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.092423 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.092398    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-780944\"/\"openshift-service-ca.crt\""
Sep 08 14:40:38.092574 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.092521    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-780944\"/\"kube-root-ca.crt\""
Sep 08 14:40:38.092643 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.092631    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-780944\"/\"default-dockercfg-vdhn5\""
Sep 08 14:40:38.092795 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.092776    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-780944\"/\"rate-test-model-780944-kserve-self-signed-certs\""
Sep 08 14:40:38.099748 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.099725    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"]
Sep 08 14:40:38.129851 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.129826    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:38.192332 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.192248    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-model-cache\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.192332 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.192294    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-home\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.192332 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.192330    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-dshm\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.192564 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.192358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh52j\" (UniqueName: \"kubernetes.io/projected/8eb29b87-e3cd-47d1-a599-f7583985b85e-kube-api-access-jh52j\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.192564 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.192394    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eb29b87-e3cd-47d1-a599-f7583985b85e-tls-certs\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.192564 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.192424    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-tmp-dir\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.293269 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.293237    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-model-cache\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.293749 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.293277    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-home\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.293749 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.293305    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-dshm\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.293749 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.293321    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh52j\" (UniqueName: \"kubernetes.io/projected/8eb29b87-e3cd-47d1-a599-f7583985b85e-kube-api-access-jh52j\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.293749 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.293352    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eb29b87-e3cd-47d1-a599-f7583985b85e-tls-certs\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.293749 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.293389    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-tmp-dir\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.293749 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.293711    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-model-cache\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.293749 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.293742    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-home\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.294090 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.293771    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-tmp-dir\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.295633 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.295607    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-dshm\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.295928 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.295909    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eb29b87-e3cd-47d1-a599-f7583985b85e-tls-certs\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.304808 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.304786    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh52j\" (UniqueName: \"kubernetes.io/projected/8eb29b87-e3cd-47d1-a599-f7583985b85e-kube-api-access-jh52j\") pod \"rate-test-model-780944-kserve-9b5984659-c26ms\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") " pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.400418 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.400383    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:38.672346 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:38.672317    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"]
Sep 08 14:40:38.674493 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:40:38.674461    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8eb29b87_e3cd_47d1_a599_f7583985b85e.slice/crio-49fbfc07b267705bd409e4e5da4eb7852273c114e284e23aa2e7e9b412ab3771 WatchSource:0}: Error finding container 49fbfc07b267705bd409e4e5da4eb7852273c114e284e23aa2e7e9b412ab3771: Status 404 returned error can't find the container with id 49fbfc07b267705bd409e4e5da4eb7852273c114e284e23aa2e7e9b412ab3771
Sep 08 14:40:39.287798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:39.287767    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56e3c0f6-4f78-4100-a718-7285998c3354" path="/var/lib/kubelet/pods/56e3c0f6-4f78-4100-a718-7285998c3354/volumes"
Sep 08 14:40:39.356054 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:39.356020    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms" event={"ID":"8eb29b87-e3cd-47d1-a599-f7583985b85e","Type":"ContainerStarted","Data":"862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4"}
Sep 08 14:40:39.356438 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:39.356062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms" event={"ID":"8eb29b87-e3cd-47d1-a599-f7583985b85e","Type":"ContainerStarted","Data":"49fbfc07b267705bd409e4e5da4eb7852273c114e284e23aa2e7e9b412ab3771"}
Sep 08 14:40:39.379850 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:39.379791    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms" podStartSLOduration=1.379769773 podStartE2EDuration="1.379769773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:40:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:40:39.375599116 +0000 UTC m=+1454.606530317" watchObservedRunningTime="2026-09-08 14:40:39.379769773 +0000 UTC m=+1454.610700975"
Sep 08 14:40:41.375484 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:41.375453    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:48.400733 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:48.400704    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:48.401191 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:48.400750    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:48.413895 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:48.413865    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:49.419333 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:49.419301    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:40:50.754295 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:50.754259    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:40:53.670410 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:53.670369    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:55.695402 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:55.695359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:58.584909 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:58.584868    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:58.686195 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:40:58.686157    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:03.405120 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:03.405076    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"]
Sep 08 14:41:03.405644 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:03.405310    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy" containerID="cri-o://4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98" gracePeriod=30
Sep 08 14:41:03.471559 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:03.471527    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:03.531809 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:03.531777    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:03.655512 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:03.655421    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:41:03.972981 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:03.972948    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:04.849169 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:04.849133    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:08.659326 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.659299    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:41:08.710673 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.710577    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-token\") pod \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") "
Sep 08 14:41:08.710673 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.710634    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-credential-socket\") pod \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") "
Sep 08 14:41:08.710673 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.710662    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-podinfo\") pod \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") "
Sep 08 14:41:08.710943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.710692    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istiod-ca-cert\") pod \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") "
Sep 08 14:41:08.710943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.710865    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-envoy\") pod \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") "
Sep 08 14:41:08.710943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.710927    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-socket\") pod \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") "
Sep 08 14:41:08.711117 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.710962    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-certs\") pod \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") "
Sep 08 14:41:08.711117 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.710999    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jlfd\" (UniqueName: \"kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-kube-api-access-4jlfd\") pod \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") "
Sep 08 14:41:08.711117 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.711002    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "06f6d56b-bfb2-42b6-93c5-065689ffcc7b" (UID: "06f6d56b-bfb2-42b6-93c5-065689ffcc7b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:41:08.711117 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.711091    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-data\") pod \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\" (UID: \"06f6d56b-bfb2-42b6-93c5-065689ffcc7b\") "
Sep 08 14:41:08.711378 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.711248    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "06f6d56b-bfb2-42b6-93c5-065689ffcc7b" (UID: "06f6d56b-bfb2-42b6-93c5-065689ffcc7b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:41:08.711378 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.711251    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "06f6d56b-bfb2-42b6-93c5-065689ffcc7b" (UID: "06f6d56b-bfb2-42b6-93c5-065689ffcc7b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:41:08.711481 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.711453    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-data" (OuterVolumeSpecName: "istio-data") pod "06f6d56b-bfb2-42b6-93c5-065689ffcc7b" (UID: "06f6d56b-bfb2-42b6-93c5-065689ffcc7b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:41:08.711547 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.711473    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "06f6d56b-bfb2-42b6-93c5-065689ffcc7b" (UID: "06f6d56b-bfb2-42b6-93c5-065689ffcc7b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:41:08.711606 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.711542    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:08.711606 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.711567    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-workload-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:08.711606 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.711587    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-data\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:08.711606 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.711599    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-credential-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:08.711810 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.711612    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istiod-ca-cert\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:08.713028 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.713000    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-token" (OuterVolumeSpecName: "istio-token") pod "06f6d56b-bfb2-42b6-93c5-065689ffcc7b" (UID: "06f6d56b-bfb2-42b6-93c5-065689ffcc7b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:41:08.713191 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.713168    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "06f6d56b-bfb2-42b6-93c5-065689ffcc7b" (UID: "06f6d56b-bfb2-42b6-93c5-065689ffcc7b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:41:08.713285 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.713267    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-kube-api-access-4jlfd" (OuterVolumeSpecName: "kube-api-access-4jlfd") pod "06f6d56b-bfb2-42b6-93c5-065689ffcc7b" (UID: "06f6d56b-bfb2-42b6-93c5-065689ffcc7b"). InnerVolumeSpecName "kube-api-access-4jlfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:41:08.713784 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.713762    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "06f6d56b-bfb2-42b6-93c5-065689ffcc7b" (UID: "06f6d56b-bfb2-42b6-93c5-065689ffcc7b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:41:08.813115 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.813078    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-token\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:08.813115 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.813110    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-podinfo\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:08.813115 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.813120    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-istio-envoy\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:08.813384 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:08.813129    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jlfd\" (UniqueName: \"kubernetes.io/projected/06f6d56b-bfb2-42b6-93c5-065689ffcc7b-kube-api-access-4jlfd\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:09.519506 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.519473    2558 generic.go:356] "Generic (PLEG): container finished" podID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerID="4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98" exitCode=0
Sep 08 14:41:09.519674 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.519522    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"
Sep 08 14:41:09.519674 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.519556    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" event={"ID":"06f6d56b-bfb2-42b6-93c5-065689ffcc7b","Type":"ContainerDied","Data":"4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98"}
Sep 08 14:41:09.519674 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.519620    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc" event={"ID":"06f6d56b-bfb2-42b6-93c5-065689ffcc7b","Type":"ContainerDied","Data":"ce2add4f38248324548d40ad77e5d163c6c65688d34a9fb4e2fe37fea851db36"}
Sep 08 14:41:09.519674 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.519649    2558 scope.go:117] "RemoveContainer" containerID="4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98"
Sep 08 14:41:09.529265 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.529068    2558 scope.go:117] "RemoveContainer" containerID="9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2"
Sep 08 14:41:09.537547 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.537527    2558 scope.go:117] "RemoveContainer" containerID="4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98"
Sep 08 14:41:09.538073 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:41:09.537927    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98\": container with ID starting with 4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98 not found: ID does not exist" containerID="4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98"
Sep 08 14:41:09.538073 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.537966    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98"} err="failed to get container status \"4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98\": rpc error: code = NotFound desc = could not find container \"4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98\": container with ID starting with 4a2b3dcdee736bbe2cc348011f55d303b19f42441ab73de8ba9ec042e64a6c98 not found: ID does not exist"
Sep 08 14:41:09.538073 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.537990    2558 scope.go:117] "RemoveContainer" containerID="9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2"
Sep 08 14:41:09.538385 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:41:09.538364    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2\": container with ID starting with 9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2 not found: ID does not exist" containerID="9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2"
Sep 08 14:41:09.538435 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.538391    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2"} err="failed to get container status \"9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2\": rpc error: code = NotFound desc = could not find container \"9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2\": container with ID starting with 9db48d43aad0ab2c5ad533aac21f2e01caa216ece611aa940b2a53467a5084f2 not found: ID does not exist"
Sep 08 14:41:09.540255 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.540231    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"]
Sep 08 14:41:09.545766 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:09.545714    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-0fd47b-openshift-default-5f6648678b-6z8dc"]
Sep 08 14:41:11.290433 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:11.290392    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" path="/var/lib/kubelet/pods/06f6d56b-bfb2-42b6-93c5-065689ffcc7b/volumes"
Sep 08 14:41:16.858558 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:16.858515    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:41:19.374705 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.374664    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"]
Sep 08 14:41:19.375251 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.374918    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725" podUID="54a395cf-9c7b-4f1c-a12f-26104db18c99" containerName="payload-processing" containerID="cri-o://a233435fc95530c5f59484b6284384fb0c38ba54789f4090ff0cee951dcc2fef" gracePeriod=30
Sep 08 14:41:19.383613 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.382988    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"]
Sep 08 14:41:19.384512 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.384084    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l" podUID="2629b93f-6c64-4a5b-abbe-8374f3fc5947" containerName="payload-pre-processing" containerID="cri-o://714ac07a43df28a37be7b07e97ad6377f080303921d6b123cf527963357559eb" gracePeriod=30
Sep 08 14:41:19.580199 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.580165    2558 generic.go:356] "Generic (PLEG): container finished" podID="2629b93f-6c64-4a5b-abbe-8374f3fc5947" containerID="714ac07a43df28a37be7b07e97ad6377f080303921d6b123cf527963357559eb" exitCode=0
Sep 08 14:41:19.580399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.580240    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l" event={"ID":"2629b93f-6c64-4a5b-abbe-8374f3fc5947","Type":"ContainerDied","Data":"714ac07a43df28a37be7b07e97ad6377f080303921d6b123cf527963357559eb"}
Sep 08 14:41:19.583447 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.583364    2558 generic.go:356] "Generic (PLEG): container finished" podID="54a395cf-9c7b-4f1c-a12f-26104db18c99" containerID="a233435fc95530c5f59484b6284384fb0c38ba54789f4090ff0cee951dcc2fef" exitCode=0
Sep 08 14:41:19.583447 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.583416    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725" event={"ID":"54a395cf-9c7b-4f1c-a12f-26104db18c99","Type":"ContainerDied","Data":"a233435fc95530c5f59484b6284384fb0c38ba54789f4090ff0cee951dcc2fef"}
Sep 08 14:41:19.659641 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.659615    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:41:19.662486 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.662462    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:41:19.830479 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.830442    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2629b93f-6c64-4a5b-abbe-8374f3fc5947-plugins-config-volume\") pod \"2629b93f-6c64-4a5b-abbe-8374f3fc5947\" (UID: \"2629b93f-6c64-4a5b-abbe-8374f3fc5947\") "
Sep 08 14:41:19.830661 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.830507    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl9ql\" (UniqueName: \"kubernetes.io/projected/2629b93f-6c64-4a5b-abbe-8374f3fc5947-kube-api-access-cl9ql\") pod \"2629b93f-6c64-4a5b-abbe-8374f3fc5947\" (UID: \"2629b93f-6c64-4a5b-abbe-8374f3fc5947\") "
Sep 08 14:41:19.830661 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.830606    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54a395cf-9c7b-4f1c-a12f-26104db18c99-plugins-config-volume\") pod \"54a395cf-9c7b-4f1c-a12f-26104db18c99\" (UID: \"54a395cf-9c7b-4f1c-a12f-26104db18c99\") "
Sep 08 14:41:19.830661 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.830634    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4gmq\" (UniqueName: \"kubernetes.io/projected/54a395cf-9c7b-4f1c-a12f-26104db18c99-kube-api-access-x4gmq\") pod \"54a395cf-9c7b-4f1c-a12f-26104db18c99\" (UID: \"54a395cf-9c7b-4f1c-a12f-26104db18c99\") "
Sep 08 14:41:19.830967 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.830921    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2629b93f-6c64-4a5b-abbe-8374f3fc5947-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2629b93f-6c64-4a5b-abbe-8374f3fc5947" (UID: "2629b93f-6c64-4a5b-abbe-8374f3fc5947"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:41:19.831066 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.831009    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54a395cf-9c7b-4f1c-a12f-26104db18c99-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "54a395cf-9c7b-4f1c-a12f-26104db18c99" (UID: "54a395cf-9c7b-4f1c-a12f-26104db18c99"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:41:19.832816 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.832791    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54a395cf-9c7b-4f1c-a12f-26104db18c99-kube-api-access-x4gmq" (OuterVolumeSpecName: "kube-api-access-x4gmq") pod "54a395cf-9c7b-4f1c-a12f-26104db18c99" (UID: "54a395cf-9c7b-4f1c-a12f-26104db18c99"). InnerVolumeSpecName "kube-api-access-x4gmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:41:19.832887 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.832833    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2629b93f-6c64-4a5b-abbe-8374f3fc5947-kube-api-access-cl9ql" (OuterVolumeSpecName: "kube-api-access-cl9ql") pod "2629b93f-6c64-4a5b-abbe-8374f3fc5947" (UID: "2629b93f-6c64-4a5b-abbe-8374f3fc5947"). InnerVolumeSpecName "kube-api-access-cl9ql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:41:19.931772 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.931691    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl9ql\" (UniqueName: \"kubernetes.io/projected/2629b93f-6c64-4a5b-abbe-8374f3fc5947-kube-api-access-cl9ql\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:19.931772 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.931721    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54a395cf-9c7b-4f1c-a12f-26104db18c99-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:19.931772 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.931731    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4gmq\" (UniqueName: \"kubernetes.io/projected/54a395cf-9c7b-4f1c-a12f-26104db18c99-kube-api-access-x4gmq\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:19.931772 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:19.931740    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2629b93f-6c64-4a5b-abbe-8374f3fc5947-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:20.589653 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:20.589619    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"
Sep 08 14:41:20.589653 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:20.589634    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l" event={"ID":"2629b93f-6c64-4a5b-abbe-8374f3fc5947","Type":"ContainerDied","Data":"777132bfc71d315b265435c7d80c7523bf91cdfe2bc49f15b39ffc64cc625fe1"}
Sep 08 14:41:20.590171 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:20.589701    2558 scope.go:117] "RemoveContainer" containerID="714ac07a43df28a37be7b07e97ad6377f080303921d6b123cf527963357559eb"
Sep 08 14:41:20.591084 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:20.591061    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"
Sep 08 14:41:20.591180 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:20.591068    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725" event={"ID":"54a395cf-9c7b-4f1c-a12f-26104db18c99","Type":"ContainerDied","Data":"b1dfd8a404918b00dcfea318982a10adaa1d559b27a125692e77595565d1ffdd"}
Sep 08 14:41:20.600742 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:20.599788    2558 scope.go:117] "RemoveContainer" containerID="a233435fc95530c5f59484b6284384fb0c38ba54789f4090ff0cee951dcc2fef"
Sep 08 14:41:20.624841 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:20.624806    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"]
Sep 08 14:41:20.627234 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:20.627189    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-41e512-58d88795bb-mjc5l"]
Sep 08 14:41:20.638145 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:20.638116    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"]
Sep 08 14:41:20.642531 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:20.642503    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41e512-85765778d5-x2725"]
Sep 08 14:41:21.287425 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:21.287395    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2629b93f-6c64-4a5b-abbe-8374f3fc5947" path="/var/lib/kubelet/pods/2629b93f-6c64-4a5b-abbe-8374f3fc5947/volumes"
Sep 08 14:41:21.287784 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:21.287772    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54a395cf-9c7b-4f1c-a12f-26104db18c99" path="/var/lib/kubelet/pods/54a395cf-9c7b-4f1c-a12f-26104db18c99/volumes"
Sep 08 14:41:21.686796 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:21.686711    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:21.788614 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:21.788578    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:23.669801 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:23.669766    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:26.090601 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.090555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:26.190393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.190356    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:26.374436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.374357    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:26.740015 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.739980    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"]
Sep 08 14:41:26.740421 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.740406    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54a395cf-9c7b-4f1c-a12f-26104db18c99" containerName="payload-processing"
Sep 08 14:41:26.740472 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.740422    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="54a395cf-9c7b-4f1c-a12f-26104db18c99" containerName="payload-processing"
Sep 08 14:41:26.740472 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.740433    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2629b93f-6c64-4a5b-abbe-8374f3fc5947" containerName="payload-pre-processing"
Sep 08 14:41:26.740472 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.740439    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2629b93f-6c64-4a5b-abbe-8374f3fc5947" containerName="payload-pre-processing"
Sep 08 14:41:26.740472 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.740452    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy"
Sep 08 14:41:26.740472 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.740459    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy"
Sep 08 14:41:26.740634 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.740525    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2629b93f-6c64-4a5b-abbe-8374f3fc5947" containerName="payload-pre-processing"
Sep 08 14:41:26.740634 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.740532    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy"
Sep 08 14:41:26.740634 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.740540    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="54a395cf-9c7b-4f1c-a12f-26104db18c99" containerName="payload-processing"
Sep 08 14:41:26.742596 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.742580    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"
Sep 08 14:41:26.745005 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.744968    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:41:26.745170 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.745091    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fddbb\""
Sep 08 14:41:26.750426 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.750387    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"]
Sep 08 14:41:26.797376 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.797341    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0600aa96-ab5d-4547-8537-48d2bf45d306-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg\" (UID: \"0600aa96-ab5d-4547-8537-48d2bf45d306\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"
Sep 08 14:41:26.850587 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.850555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:26.898115 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.898081    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0600aa96-ab5d-4547-8537-48d2bf45d306-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg\" (UID: \"0600aa96-ab5d-4547-8537-48d2bf45d306\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"
Sep 08 14:41:26.898776 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:26.898754    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0600aa96-ab5d-4547-8537-48d2bf45d306-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg\" (UID: \"0600aa96-ab5d-4547-8537-48d2bf45d306\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"
Sep 08 14:41:27.054635 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:27.054546    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"
Sep 08 14:41:27.325742 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:27.325717    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"]
Sep 08 14:41:27.327698 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:41:27.327658    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0600aa96_ab5d_4547_8537_48d2bf45d306.slice/crio-03d6b11348ba758bf6bd9bf3b59e2663a2282dd0f32991a2c507019d4f5c279d WatchSource:0}: Error finding container 03d6b11348ba758bf6bd9bf3b59e2663a2282dd0f32991a2c507019d4f5c279d: Status 404 returned error can't find the container with id 03d6b11348ba758bf6bd9bf3b59e2663a2282dd0f32991a2c507019d4f5c279d
Sep 08 14:41:27.329704 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:27.329687    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:41:27.633513 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:27.633421    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg" event={"ID":"0600aa96-ab5d-4547-8537-48d2bf45d306","Type":"ContainerStarted","Data":"03d6b11348ba758bf6bd9bf3b59e2663a2282dd0f32991a2c507019d4f5c279d"}
Sep 08 14:41:30.649904 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:30.649805    2558 generic.go:356] "Generic (PLEG): container finished" podID="0600aa96-ab5d-4547-8537-48d2bf45d306" containerID="f038d6e340ec02dcd6e296bec62dd18454cfe143d72da9d4b4dd01bc91fc11f2" exitCode=0
Sep 08 14:41:30.650301 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:30.649922    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg" event={"ID":"0600aa96-ab5d-4547-8537-48d2bf45d306","Type":"ContainerDied","Data":"f038d6e340ec02dcd6e296bec62dd18454cfe143d72da9d4b4dd01bc91fc11f2"}
Sep 08 14:41:31.783692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:31.783665    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"
Sep 08 14:41:31.849871 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:31.849837    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0600aa96-ab5d-4547-8537-48d2bf45d306-maas-api-service-ca\") pod \"0600aa96-ab5d-4547-8537-48d2bf45d306\" (UID: \"0600aa96-ab5d-4547-8537-48d2bf45d306\") "
Sep 08 14:41:31.850301 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:31.850278    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0600aa96-ab5d-4547-8537-48d2bf45d306-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0600aa96-ab5d-4547-8537-48d2bf45d306" (UID: "0600aa96-ab5d-4547-8537-48d2bf45d306"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:41:31.951038 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:31.950947    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0600aa96-ab5d-4547-8537-48d2bf45d306-maas-api-service-ca\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:32.558270 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:32.558236    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:41:32.662416 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:32.662386    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"
Sep 08 14:41:32.662416 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:32.662401    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg" event={"ID":"0600aa96-ab5d-4547-8537-48d2bf45d306","Type":"ContainerDied","Data":"03d6b11348ba758bf6bd9bf3b59e2663a2282dd0f32991a2c507019d4f5c279d"}
Sep 08 14:41:32.662615 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:32.662444    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03d6b11348ba758bf6bd9bf3b59e2663a2282dd0f32991a2c507019d4f5c279d"
Sep 08 14:41:37.376190 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.376154    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:37.410286 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.410249    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"]
Sep 08 14:41:37.410606 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.410568    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2" podUID="8d08211f-12b3-4cf7-9a22-26aa1bff471b" containerName="payload-pre-processing" containerID="cri-o://1b724d65871cf44ee4c913f0c957ba060b64928b8a5e44d4299f0f5a59dd5c72" gracePeriod=30
Sep 08 14:41:37.479142 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.479107    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:37.576632 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.576597    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:37.587428 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.587392    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"]
Sep 08 14:41:37.590485 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.590452    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-23a1d4-792kg"]
Sep 08 14:41:37.677674 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.677640    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:37.690139 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.690105    2558 generic.go:356] "Generic (PLEG): container finished" podID="8d08211f-12b3-4cf7-9a22-26aa1bff471b" containerID="1b724d65871cf44ee4c913f0c957ba060b64928b8a5e44d4299f0f5a59dd5c72" exitCode=0
Sep 08 14:41:37.690328 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.690309    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2" event={"ID":"8d08211f-12b3-4cf7-9a22-26aa1bff471b","Type":"ContainerDied","Data":"1b724d65871cf44ee4c913f0c957ba060b64928b8a5e44d4299f0f5a59dd5c72"}
Sep 08 14:41:37.690411 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.690398    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2" event={"ID":"8d08211f-12b3-4cf7-9a22-26aa1bff471b","Type":"ContainerDied","Data":"24225d01371087d31633f6caac4250577a0f3222820fdd29d5350824d340f893"}
Sep 08 14:41:37.690468 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.690421    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24225d01371087d31633f6caac4250577a0f3222820fdd29d5350824d340f893"
Sep 08 14:41:37.700593 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.700561    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:41:37.808481 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.808443    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d08211f-12b3-4cf7-9a22-26aa1bff471b-plugins-config-volume\") pod \"8d08211f-12b3-4cf7-9a22-26aa1bff471b\" (UID: \"8d08211f-12b3-4cf7-9a22-26aa1bff471b\") "
Sep 08 14:41:37.808694 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.808505    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlfb2\" (UniqueName: \"kubernetes.io/projected/8d08211f-12b3-4cf7-9a22-26aa1bff471b-kube-api-access-dlfb2\") pod \"8d08211f-12b3-4cf7-9a22-26aa1bff471b\" (UID: \"8d08211f-12b3-4cf7-9a22-26aa1bff471b\") "
Sep 08 14:41:37.808925 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.808900    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d08211f-12b3-4cf7-9a22-26aa1bff471b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8d08211f-12b3-4cf7-9a22-26aa1bff471b" (UID: "8d08211f-12b3-4cf7-9a22-26aa1bff471b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:41:37.810766 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.810741    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d08211f-12b3-4cf7-9a22-26aa1bff471b-kube-api-access-dlfb2" (OuterVolumeSpecName: "kube-api-access-dlfb2") pod "8d08211f-12b3-4cf7-9a22-26aa1bff471b" (UID: "8d08211f-12b3-4cf7-9a22-26aa1bff471b"). InnerVolumeSpecName "kube-api-access-dlfb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:41:37.909685 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.909646    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlfb2\" (UniqueName: \"kubernetes.io/projected/8d08211f-12b3-4cf7-9a22-26aa1bff471b-kube-api-access-dlfb2\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:37.909685 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.909679    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d08211f-12b3-4cf7-9a22-26aa1bff471b-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:37.985378 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:37.985342    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:38.695179 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:38.695147    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"
Sep 08 14:41:38.720566 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:38.720534    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"]
Sep 08 14:41:38.722272 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:38.722250    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-23a1d4-6d86ccd746-kskk2"]
Sep 08 14:41:39.287555 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:39.287523    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0600aa96-ab5d-4547-8537-48d2bf45d306" path="/var/lib/kubelet/pods/0600aa96-ab5d-4547-8537-48d2bf45d306/volumes"
Sep 08 14:41:39.287935 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:39.287921    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d08211f-12b3-4cf7-9a22-26aa1bff471b" path="/var/lib/kubelet/pods/8d08211f-12b3-4cf7-9a22-26aa1bff471b/volumes"
Sep 08 14:41:44.129153 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:44.129114    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:44.190307 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:44.190272    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:44.405439 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:44.405344    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"]
Sep 08 14:41:44.405688 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:44.405660    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy" containerID="cri-o://cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4" gracePeriod=30
Sep 08 14:41:44.433380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:44.433344    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:44.850343 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:44.850307    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:41:44.987823 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:44.987792    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:49.663875 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.663842    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:41:49.708937 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.708846    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"]
Sep 08 14:41:49.727331 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727293    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-envoy\") pod \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") "
Sep 08 14:41:49.727529 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727346    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-certs\") pod \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") "
Sep 08 14:41:49.727529 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727410    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-podinfo\") pod \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") "
Sep 08 14:41:49.727529 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727445    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztnnr\" (UniqueName: \"kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-kube-api-access-ztnnr\") pod \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") "
Sep 08 14:41:49.727529 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727473    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-credential-socket\") pod \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") "
Sep 08 14:41:49.727529 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727522    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istiod-ca-cert\") pod \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") "
Sep 08 14:41:49.727799 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727603    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-socket\") pod \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") "
Sep 08 14:41:49.727799 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727598    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d07d5b5b-6a9b-44c5-bd00-fd9084282e66" (UID: "d07d5b5b-6a9b-44c5-bd00-fd9084282e66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:41:49.727799 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727642    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-token\") pod \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") "
Sep 08 14:41:49.727799 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727669    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-data\") pod \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\" (UID: \"d07d5b5b-6a9b-44c5-bd00-fd9084282e66\") "
Sep 08 14:41:49.728005 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727853    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d07d5b5b-6a9b-44c5-bd00-fd9084282e66" (UID: "d07d5b5b-6a9b-44c5-bd00-fd9084282e66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:41:49.728005 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.727861    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:49.728118 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.728085    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-data" (OuterVolumeSpecName: "istio-data") pod "d07d5b5b-6a9b-44c5-bd00-fd9084282e66" (UID: "d07d5b5b-6a9b-44c5-bd00-fd9084282e66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:41:49.728692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.728409    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d07d5b5b-6a9b-44c5-bd00-fd9084282e66" (UID: "d07d5b5b-6a9b-44c5-bd00-fd9084282e66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:41:49.728692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.728569    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-data\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:49.728692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.728589    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:49.728692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.728603    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-credential-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:49.728692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.728646    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d07d5b5b-6a9b-44c5-bd00-fd9084282e66" (UID: "d07d5b5b-6a9b-44c5-bd00-fd9084282e66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:41:49.728692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.728695    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istiod-ca-cert\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:49.729895 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.729871    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d07d5b5b-6a9b-44c5-bd00-fd9084282e66" (UID: "d07d5b5b-6a9b-44c5-bd00-fd9084282e66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:41:49.730158 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.730135    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d07d5b5b-6a9b-44c5-bd00-fd9084282e66" (UID: "d07d5b5b-6a9b-44c5-bd00-fd9084282e66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:41:49.730276 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.730248    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-kube-api-access-ztnnr" (OuterVolumeSpecName: "kube-api-access-ztnnr") pod "d07d5b5b-6a9b-44c5-bd00-fd9084282e66" (UID: "d07d5b5b-6a9b-44c5-bd00-fd9084282e66"). InnerVolumeSpecName "kube-api-access-ztnnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:41:49.730416 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.730398    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-token" (OuterVolumeSpecName: "istio-token") pod "d07d5b5b-6a9b-44c5-bd00-fd9084282e66" (UID: "d07d5b5b-6a9b-44c5-bd00-fd9084282e66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:41:49.751284 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.751240    2558 generic.go:356] "Generic (PLEG): container finished" podID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerID="cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4" exitCode=0
Sep 08 14:41:49.751284 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.751275    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"
Sep 08 14:41:49.751536 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.751297    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" event={"ID":"d07d5b5b-6a9b-44c5-bd00-fd9084282e66","Type":"ContainerDied","Data":"cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4"}
Sep 08 14:41:49.751536 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.751352    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q" event={"ID":"d07d5b5b-6a9b-44c5-bd00-fd9084282e66","Type":"ContainerDied","Data":"656998680f16330ffafca0a6d30343d9d2de941a853ccb62426bbb0a31a1090b"}
Sep 08 14:41:49.751536 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.751376    2558 scope.go:117] "RemoveContainer" containerID="cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4"
Sep 08 14:41:49.761440 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.761419    2558 scope.go:117] "RemoveContainer" containerID="87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90"
Sep 08 14:41:49.770380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.770360    2558 scope.go:117] "RemoveContainer" containerID="cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4"
Sep 08 14:41:49.770691 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:41:49.770670    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4\": container with ID starting with cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4 not found: ID does not exist" containerID="cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4"
Sep 08 14:41:49.770744 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.770701    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4"} err="failed to get container status \"cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4\": rpc error: code = NotFound desc = could not find container \"cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4\": container with ID starting with cdca8cf672a9ccd8fbaf22a4db2be98b691b760f473aa229e29d9ad9d95a18c4 not found: ID does not exist"
Sep 08 14:41:49.770744 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.770720    2558 scope.go:117] "RemoveContainer" containerID="87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90"
Sep 08 14:41:49.771005 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:41:49.770987    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90\": container with ID starting with 87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90 not found: ID does not exist" containerID="87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90"
Sep 08 14:41:49.771067 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.771010    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90"} err="failed to get container status \"87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90\": rpc error: code = NotFound desc = could not find container \"87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90\": container with ID starting with 87294183a048995d23c5c2470f3e36c09875f39c2812e882c00e73922d340c90 not found: ID does not exist"
Sep 08 14:41:49.775662 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.775637    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"]
Sep 08 14:41:49.780374 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.780346    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-23a1d4-openshift-default-5b95b497d8-d5w7q"]
Sep 08 14:41:49.829950 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.829912    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-podinfo\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:49.829950 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.829944    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztnnr\" (UniqueName: \"kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-kube-api-access-ztnnr\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:49.829950 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.829956    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-workload-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:49.830266 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.829967    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-token\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:49.830266 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:49.829975    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d07d5b5b-6a9b-44c5-bd00-fd9084282e66-istio-envoy\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:50.851436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:50.851404    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:51.287885 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.287848    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" path="/var/lib/kubelet/pods/d07d5b5b-6a9b-44c5-bd00-fd9084282e66/volumes"
Sep 08 14:41:51.631070 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.630986    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"]
Sep 08 14:41:51.631487 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631472    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy"
Sep 08 14:41:51.631554 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631489    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy"
Sep 08 14:41:51.631554 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631500    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d08211f-12b3-4cf7-9a22-26aa1bff471b" containerName="payload-pre-processing"
Sep 08 14:41:51.631554 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631506    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d08211f-12b3-4cf7-9a22-26aa1bff471b" containerName="payload-pre-processing"
Sep 08 14:41:51.631554 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631515    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy"
Sep 08 14:41:51.631554 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631521    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy"
Sep 08 14:41:51.631554 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631544    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0600aa96-ab5d-4547-8537-48d2bf45d306" containerName="revoke-keys"
Sep 08 14:41:51.631554 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631550    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0600aa96-ab5d-4547-8537-48d2bf45d306" containerName="revoke-keys"
Sep 08 14:41:51.631798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631611    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="06f6d56b-bfb2-42b6-93c5-065689ffcc7b" containerName="istio-proxy"
Sep 08 14:41:51.631798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631625    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d08211f-12b3-4cf7-9a22-26aa1bff471b" containerName="payload-pre-processing"
Sep 08 14:41:51.631798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631633    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy"
Sep 08 14:41:51.631798 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.631639    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0600aa96-ab5d-4547-8537-48d2bf45d306" containerName="revoke-keys"
Sep 08 14:41:51.636370 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.636342    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.640999 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.640975    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-274ffa-openshift-default-dockercfg-flqf7\""
Sep 08 14:41:51.659491 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.659452    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"]
Sep 08 14:41:51.751340 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.751302    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14f24731-0f42-4a1e-8d8f-d58131befb0e-istiod-ca-cert\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.751517 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.751355    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-socket\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.751517 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.751414    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrfgm\" (UniqueName: \"kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-kube-api-access-jrfgm\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.751626 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.751519    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-certs\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.751626 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.751538    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-token\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.751626 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.751557    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-data\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.751626 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.751591    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-envoy\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.751626 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.751615    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-credential-socket\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.751791 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.751640    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-podinfo\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.852506 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.852463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-certs\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.852506 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.852501    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-token\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.852518    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-data\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.852550    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-envoy\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.852609    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-credential-socket\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.852656    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-podinfo\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.852725    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14f24731-0f42-4a1e-8d8f-d58131befb0e-istiod-ca-cert\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.852754    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-socket\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.852776    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrfgm\" (UniqueName: \"kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-kube-api-access-jrfgm\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.852907    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-certs\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853381 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.853116    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-credential-socket\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853381 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.853190    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-socket\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853381 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.853248    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-data\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.853479 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.853456    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14f24731-0f42-4a1e-8d8f-d58131befb0e-istiod-ca-cert\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.854988 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.854966    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-envoy\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.855306 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.855288    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-podinfo\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.862100 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.862077    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-token\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.862353 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.862333    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrfgm\" (UniqueName: \"kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-kube-api-access-jrfgm\") pod \"e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") " pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:51.947485 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:51.947379    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:52.225779 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:52.225752    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"]
Sep 08 14:41:52.227704 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:41:52.227669    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f24731_0f42_4a1e_8d8f_d58131befb0e.slice/crio-9eae57afa3a4e440e52939b08cea18d01a81bfdeabeeb99bf21a8e9303c4c9dd WatchSource:0}: Error finding container 9eae57afa3a4e440e52939b08cea18d01a81bfdeabeeb99bf21a8e9303c4c9dd: Status 404 returned error can't find the container with id 9eae57afa3a4e440e52939b08cea18d01a81bfdeabeeb99bf21a8e9303c4c9dd
Sep 08 14:41:52.229898 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:52.229864    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:41:52.229997 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:52.229946    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:41:52.230063 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:52.229996    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:41:52.769980 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:52.769939    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" event={"ID":"14f24731-0f42-4a1e-8d8f-d58131befb0e","Type":"ContainerStarted","Data":"113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64"}
Sep 08 14:41:52.770162 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:52.770000    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" event={"ID":"14f24731-0f42-4a1e-8d8f-d58131befb0e","Type":"ContainerStarted","Data":"9eae57afa3a4e440e52939b08cea18d01a81bfdeabeeb99bf21a8e9303c4c9dd"}
Sep 08 14:41:52.793112 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:52.793053    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" podStartSLOduration=1.793034729 podStartE2EDuration="1.793034729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:41:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:41:52.789965945 +0000 UTC m=+1528.020897142" watchObservedRunningTime="2026-09-08 14:41:52.793034729 +0000 UTC m=+1528.023965918"
Sep 08 14:41:52.947893 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:52.947851    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:53.194865 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:53.194767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:53.289737 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:53.289703    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:53.953143 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:53.953109    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:54.696372 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:54.696336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:54.781124 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:54.781090    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:54.782234 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:54.782174    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:41:54.795829 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:54.795800    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:55.457356 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.457321    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:41:55.487609 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.487573    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:55.515393 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.515360    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"]
Sep 08 14:41:55.515655 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.515626    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6" podUID="ba6c4672-5908-4b5d-a2eb-86ac24fee986" containerName="payload-processing" containerID="cri-o://54fa355cc8034820d5f7a6d6fef232caca1aa18523d66d53a17bcd3619f5c521" gracePeriod=30
Sep 08 14:41:55.587910 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.587851    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:55.688514 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.688224    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:55.787300 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.787267    2558 generic.go:356] "Generic (PLEG): container finished" podID="ba6c4672-5908-4b5d-a2eb-86ac24fee986" containerID="54fa355cc8034820d5f7a6d6fef232caca1aa18523d66d53a17bcd3619f5c521" exitCode=0
Sep 08 14:41:55.787430 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.787367    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6" event={"ID":"ba6c4672-5908-4b5d-a2eb-86ac24fee986","Type":"ContainerDied","Data":"54fa355cc8034820d5f7a6d6fef232caca1aa18523d66d53a17bcd3619f5c521"}
Sep 08 14:41:55.787430 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.787421    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6" event={"ID":"ba6c4672-5908-4b5d-a2eb-86ac24fee986","Type":"ContainerDied","Data":"f80e0eee740136a855d1372fae33c9c521d8d3b1a06b8e3d87953e090518aaba"}
Sep 08 14:41:55.787518 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.787438    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f80e0eee740136a855d1372fae33c9c521d8d3b1a06b8e3d87953e090518aaba"
Sep 08 14:41:55.788197 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.788175    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:41:55.788507 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.788488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:55.889666 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.889628    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:55.895852 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.895823    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba6c4672-5908-4b5d-a2eb-86ac24fee986-plugins-config-volume\") pod \"ba6c4672-5908-4b5d-a2eb-86ac24fee986\" (UID: \"ba6c4672-5908-4b5d-a2eb-86ac24fee986\") "
Sep 08 14:41:55.896009 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.895923    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfqtr\" (UniqueName: \"kubernetes.io/projected/ba6c4672-5908-4b5d-a2eb-86ac24fee986-kube-api-access-jfqtr\") pod \"ba6c4672-5908-4b5d-a2eb-86ac24fee986\" (UID: \"ba6c4672-5908-4b5d-a2eb-86ac24fee986\") "
Sep 08 14:41:55.896308 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.896270    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba6c4672-5908-4b5d-a2eb-86ac24fee986-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ba6c4672-5908-4b5d-a2eb-86ac24fee986" (UID: "ba6c4672-5908-4b5d-a2eb-86ac24fee986"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:41:55.896412 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.896382    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba6c4672-5908-4b5d-a2eb-86ac24fee986-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:55.898041 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.898021    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba6c4672-5908-4b5d-a2eb-86ac24fee986-kube-api-access-jfqtr" (OuterVolumeSpecName: "kube-api-access-jfqtr") pod "ba6c4672-5908-4b5d-a2eb-86ac24fee986" (UID: "ba6c4672-5908-4b5d-a2eb-86ac24fee986"). InnerVolumeSpecName "kube-api-access-jfqtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:41:55.991414 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.991382    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:55.997679 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:55.997650    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfqtr\" (UniqueName: \"kubernetes.io/projected/ba6c4672-5908-4b5d-a2eb-86ac24fee986-kube-api-access-jfqtr\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:41:56.090108 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.090015    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:56.191103 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.191063    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:56.484618 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.484578    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:56.584311 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.584270    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:56.585102 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.585080    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"]
Sep 08 14:41:56.585505 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.585492    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba6c4672-5908-4b5d-a2eb-86ac24fee986" containerName="payload-processing"
Sep 08 14:41:56.585546 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.585508    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba6c4672-5908-4b5d-a2eb-86ac24fee986" containerName="payload-processing"
Sep 08 14:41:56.585546 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.585528    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy"
Sep 08 14:41:56.585546 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.585534    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy"
Sep 08 14:41:56.585683 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.585593    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba6c4672-5908-4b5d-a2eb-86ac24fee986" containerName="payload-processing"
Sep 08 14:41:56.585683 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.585602    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d07d5b5b-6a9b-44c5-bd00-fd9084282e66" containerName="istio-proxy"
Sep 08 14:41:56.588572 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.588551    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:41:56.591422 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.591401    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-274ffa-dockercfg-vt5xd\""
Sep 08 14:41:56.591520 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.591405    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-274ffa\""
Sep 08 14:41:56.599279 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.599252    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"]
Sep 08 14:41:56.705372 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.705334    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c821f86-fb94-4c55-8675-53c333662759-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p\" (UID: \"6c821f86-fb94-4c55-8675-53c333662759\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:41:56.705552 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.705491    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcdgg\" (UniqueName: \"kubernetes.io/projected/6c821f86-fb94-4c55-8675-53c333662759-kube-api-access-bcdgg\") pod \"payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p\" (UID: \"6c821f86-fb94-4c55-8675-53c333662759\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:41:56.792881 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.792792    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"
Sep 08 14:41:56.806633 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.806600    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcdgg\" (UniqueName: \"kubernetes.io/projected/6c821f86-fb94-4c55-8675-53c333662759-kube-api-access-bcdgg\") pod \"payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p\" (UID: \"6c821f86-fb94-4c55-8675-53c333662759\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:41:56.806780 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.806686    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c821f86-fb94-4c55-8675-53c333662759-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p\" (UID: \"6c821f86-fb94-4c55-8675-53c333662759\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:41:56.807415 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.807396    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c821f86-fb94-4c55-8675-53c333662759-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p\" (UID: \"6c821f86-fb94-4c55-8675-53c333662759\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:41:56.815070 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.815039    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"]
Sep 08 14:41:56.817466 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.817443    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-34656b-7c6c67b96c-g59m6"]
Sep 08 14:41:56.818829 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.818811    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcdgg\" (UniqueName: \"kubernetes.io/projected/6c821f86-fb94-4c55-8675-53c333662759-kube-api-access-bcdgg\") pod \"payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p\" (UID: \"6c821f86-fb94-4c55-8675-53c333662759\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:41:56.900650 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:56.900599    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:41:57.176435 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:57.176403    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"]
Sep 08 14:41:57.288826 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:57.288790    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba6c4672-5908-4b5d-a2eb-86ac24fee986" path="/var/lib/kubelet/pods/ba6c4672-5908-4b5d-a2eb-86ac24fee986/volumes"
Sep 08 14:41:57.799364 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:57.799321    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p" event={"ID":"6c821f86-fb94-4c55-8675-53c333662759","Type":"ContainerStarted","Data":"acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111"}
Sep 08 14:41:57.799788 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:57.799379    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p" event={"ID":"6c821f86-fb94-4c55-8675-53c333662759","Type":"ContainerStarted","Data":"7ebcdfe6e65dd0360220e9342573d5b6ae672083f22db2f41ca84228c51499a9"}
Sep 08 14:41:57.799788 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:57.799400    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:41:57.816633 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:57.816580    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p" podStartSLOduration=1.8165654199999999 podStartE2EDuration="1.81656542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:41:57.814693694 +0000 UTC m=+1533.045624890" watchObservedRunningTime="2026-09-08 14:41:57.81656542 +0000 UTC m=+1533.047496659"
Sep 08 14:41:58.121817 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:58.121729    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:58.469019 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:41:58.468935    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:04.043952 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:04.043904    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:04.729402 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:04.729362    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms" podUID="8eb29b87-e3cd-47d1-a599-f7583985b85e" containerName="main" containerID="cri-o://862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4" gracePeriod=45
Sep 08 14:42:05.040326 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.040296    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:42:05.087733 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.087692    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-model-cache\") pod \"8eb29b87-e3cd-47d1-a599-f7583985b85e\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") "
Sep 08 14:42:05.088185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.087782    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-dshm\") pod \"8eb29b87-e3cd-47d1-a599-f7583985b85e\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") "
Sep 08 14:42:05.088185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.087819    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-home\") pod \"8eb29b87-e3cd-47d1-a599-f7583985b85e\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") "
Sep 08 14:42:05.088185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.087862    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-tmp-dir\") pod \"8eb29b87-e3cd-47d1-a599-f7583985b85e\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") "
Sep 08 14:42:05.088185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.087899    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jh52j\" (UniqueName: \"kubernetes.io/projected/8eb29b87-e3cd-47d1-a599-f7583985b85e-kube-api-access-jh52j\") pod \"8eb29b87-e3cd-47d1-a599-f7583985b85e\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") "
Sep 08 14:42:05.088185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.087929    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eb29b87-e3cd-47d1-a599-f7583985b85e-tls-certs\") pod \"8eb29b87-e3cd-47d1-a599-f7583985b85e\" (UID: \"8eb29b87-e3cd-47d1-a599-f7583985b85e\") "
Sep 08 14:42:05.088185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.087955    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-model-cache" (OuterVolumeSpecName: "model-cache") pod "8eb29b87-e3cd-47d1-a599-f7583985b85e" (UID: "8eb29b87-e3cd-47d1-a599-f7583985b85e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:42:05.088185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.088085    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-home" (OuterVolumeSpecName: "home") pod "8eb29b87-e3cd-47d1-a599-f7583985b85e" (UID: "8eb29b87-e3cd-47d1-a599-f7583985b85e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:42:05.088185 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.088102    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8eb29b87-e3cd-47d1-a599-f7583985b85e" (UID: "8eb29b87-e3cd-47d1-a599-f7583985b85e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:42:05.088589 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.088338    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-home\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:42:05.088589 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.088358    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-tmp-dir\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:42:05.088589 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.088372    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-model-cache\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:42:05.090031 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.089999    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eb29b87-e3cd-47d1-a599-f7583985b85e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8eb29b87-e3cd-47d1-a599-f7583985b85e" (UID: "8eb29b87-e3cd-47d1-a599-f7583985b85e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:42:05.090031 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.090012    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-dshm" (OuterVolumeSpecName: "dshm") pod "8eb29b87-e3cd-47d1-a599-f7583985b85e" (UID: "8eb29b87-e3cd-47d1-a599-f7583985b85e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:42:05.090173 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.090041    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eb29b87-e3cd-47d1-a599-f7583985b85e-kube-api-access-jh52j" (OuterVolumeSpecName: "kube-api-access-jh52j") pod "8eb29b87-e3cd-47d1-a599-f7583985b85e" (UID: "8eb29b87-e3cd-47d1-a599-f7583985b85e"). InnerVolumeSpecName "kube-api-access-jh52j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:42:05.189339 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.189293    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eb29b87-e3cd-47d1-a599-f7583985b85e-dshm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:42:05.189339 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.189330    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jh52j\" (UniqueName: \"kubernetes.io/projected/8eb29b87-e3cd-47d1-a599-f7583985b85e-kube-api-access-jh52j\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:42:05.189339 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.189344    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eb29b87-e3cd-47d1-a599-f7583985b85e-tls-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:42:05.212386 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.212348    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:05.847944 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.847890    2558 generic.go:356] "Generic (PLEG): container finished" podID="8eb29b87-e3cd-47d1-a599-f7583985b85e" containerID="862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4" exitCode=0
Sep 08 14:42:05.848131 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.847992    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"
Sep 08 14:42:05.848750 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.848012    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms" event={"ID":"8eb29b87-e3cd-47d1-a599-f7583985b85e","Type":"ContainerDied","Data":"862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4"}
Sep 08 14:42:05.848901 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.848776    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms" event={"ID":"8eb29b87-e3cd-47d1-a599-f7583985b85e","Type":"ContainerDied","Data":"49fbfc07b267705bd409e4e5da4eb7852273c114e284e23aa2e7e9b412ab3771"}
Sep 08 14:42:05.848901 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.848807    2558 scope.go:117] "RemoveContainer" containerID="862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4"
Sep 08 14:42:05.865657 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.865630    2558 scope.go:117] "RemoveContainer" containerID="862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4"
Sep 08 14:42:05.866061 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:42:05.866028    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4\": container with ID starting with 862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4 not found: ID does not exist" containerID="862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4"
Sep 08 14:42:05.866165 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.866074    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4"} err="failed to get container status \"862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4\": rpc error: code = NotFound desc = could not find container \"862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4\": container with ID starting with 862120735f0fdf8f7b55b6f6933d44c27a79942ce1a5ffd93b3e29f34206ade4 not found: ID does not exist"
Sep 08 14:42:05.887093 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.887057    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"]
Sep 08 14:42:05.891576 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:05.891547    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-780944/rate-test-model-780944-kserve-9b5984659-c26ms"]
Sep 08 14:42:07.055415 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:07.055370    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:42:07.287384 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:07.287351    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eb29b87-e3cd-47d1-a599-f7583985b85e" path="/var/lib/kubelet/pods/8eb29b87-e3cd-47d1-a599-f7583985b85e/volumes"
Sep 08 14:42:08.806786 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:08.806752    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:42:10.823398 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:10.823355    2558 patch_prober.go:28] interesting pod/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 14:42:10.823808 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:10.823421    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 14:42:12.899448 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:12.899409    2558 generic.go:356] "Generic (PLEG): container finished" podID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerID="113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64" exitCode=0
Sep 08 14:42:12.899864 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:12.899479    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" event={"ID":"14f24731-0f42-4a1e-8d8f-d58131befb0e","Type":"ContainerDied","Data":"113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64"}
Sep 08 14:42:12.899927 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:12.899905    2558 scope.go:117] "RemoveContainer" containerID="113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64"
Sep 08 14:42:12.900976 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:12.900946    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:42:12.901068 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:12.901010    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:42:12.901068 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:12.901042    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 08 14:42:13.906100 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:13.906060    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" event={"ID":"14f24731-0f42-4a1e-8d8f-d58131befb0e","Type":"ContainerStarted","Data":"80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74"}
Sep 08 14:42:13.948305 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:13.948262    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:42:13.948655 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:13.948626    2558 patch_prober.go:28] interesting pod/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 08 14:42:13.948755 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:13.948688    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 08 14:42:14.948066 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:14.948020    2558 patch_prober.go:28] interesting pod/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 08 14:42:14.948703 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:14.948665    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 08 14:42:15.948192 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:15.948146    2558 patch_prober.go:28] interesting pod/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 08 14:42:15.948818 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:15.948782    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 08 14:42:16.886436 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:16.886395    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:16.948859 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:16.948816    2558 patch_prober.go:28] interesting pod/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 08 14:42:16.949473 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:16.949437    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 08 14:42:17.948857 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:17.948811    2558 patch_prober.go:28] interesting pod/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 08 14:42:17.949076 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:17.948883    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 08 14:42:18.948022 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:18.947977    2558 patch_prober.go:28] interesting pod/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 08 14:42:18.948247 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:18.948045    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 08 14:42:19.791576 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:19.791535    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:19.951713 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:19.951684    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:42:19.951978 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:19.951948    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:42:19.952754 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:19.952734    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:42:20.496286 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:20.496254    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:20.817098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:20.817007    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:21.030344 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:21.030308    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:23.676832 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:23.676801    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:23.779652 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:23.779618    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:24.976446 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:24.976406    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:25.076782 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.076750    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:25.176891 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.176855    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:25.459859 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.459823    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:25.495592 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.495563    2558 scope.go:117] "RemoveContainer" containerID="54fa355cc8034820d5f7a6d6fef232caca1aa18523d66d53a17bcd3619f5c521"
Sep 08 14:42:25.504312 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.504197    2558 scope.go:117] "RemoveContainer" containerID="1b724d65871cf44ee4c913f0c957ba060b64928b8a5e44d4299f0f5a59dd5c72"
Sep 08 14:42:25.535934 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.535908    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:25.590540 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.590507    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"]
Sep 08 14:42:25.591033 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.591005    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eb29b87-e3cd-47d1-a599-f7583985b85e" containerName="main"
Sep 08 14:42:25.591033 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.591025    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eb29b87-e3cd-47d1-a599-f7583985b85e" containerName="main"
Sep 08 14:42:25.591262 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.591105    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eb29b87-e3cd-47d1-a599-f7583985b85e" containerName="main"
Sep 08 14:42:25.595836 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.595816    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:42:25.598310 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.598290    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f7d00a\""
Sep 08 14:42:25.598538 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.598522    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f7d00a-dockercfg-9225w\""
Sep 08 14:42:25.611280 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.611252    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"]
Sep 08 14:42:25.695391 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.695358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca9504a-7456-4cb1-b202-a976227bb406-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx\" (UID: \"8ca9504a-7456-4cb1-b202-a976227bb406\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:42:25.695587 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.695513    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hdpv\" (UniqueName: \"kubernetes.io/projected/8ca9504a-7456-4cb1-b202-a976227bb406-kube-api-access-9hdpv\") pod \"payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx\" (UID: \"8ca9504a-7456-4cb1-b202-a976227bb406\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:42:25.796616 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.796578    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca9504a-7456-4cb1-b202-a976227bb406-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx\" (UID: \"8ca9504a-7456-4cb1-b202-a976227bb406\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:42:25.796824 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.796662    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hdpv\" (UniqueName: \"kubernetes.io/projected/8ca9504a-7456-4cb1-b202-a976227bb406-kube-api-access-9hdpv\") pod \"payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx\" (UID: \"8ca9504a-7456-4cb1-b202-a976227bb406\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:42:25.797410 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.797390    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca9504a-7456-4cb1-b202-a976227bb406-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx\" (UID: \"8ca9504a-7456-4cb1-b202-a976227bb406\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:42:25.805643 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.805609    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hdpv\" (UniqueName: \"kubernetes.io/projected/8ca9504a-7456-4cb1-b202-a976227bb406-kube-api-access-9hdpv\") pod \"payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx\" (UID: \"8ca9504a-7456-4cb1-b202-a976227bb406\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:42:25.906195 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:25.906154    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:42:26.176422 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:26.176317    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"]
Sep 08 14:42:26.179850 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:42:26.179812    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca9504a_7456_4cb1_b202_a976227bb406.slice/crio-d75be741fe84d18ac3b6f4d1906c2c9850e0c929189a43d67f350249daed5b99 WatchSource:0}: Error finding container d75be741fe84d18ac3b6f4d1906c2c9850e0c929189a43d67f350249daed5b99: Status 404 returned error can't find the container with id d75be741fe84d18ac3b6f4d1906c2c9850e0c929189a43d67f350249daed5b99
Sep 08 14:42:26.979935 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:26.979895    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx" event={"ID":"8ca9504a-7456-4cb1-b202-a976227bb406","Type":"ContainerStarted","Data":"03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e"}
Sep 08 14:42:26.980127 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:26.979988    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx" event={"ID":"8ca9504a-7456-4cb1-b202-a976227bb406","Type":"ContainerStarted","Data":"d75be741fe84d18ac3b6f4d1906c2c9850e0c929189a43d67f350249daed5b99"}
Sep 08 14:42:26.980127 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:26.980107    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:42:26.996732 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:26.996676    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx" podStartSLOduration=1.996660224 podStartE2EDuration="1.996660224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:42:26.995242939 +0000 UTC m=+1562.226174139" watchObservedRunningTime="2026-09-08 14:42:26.996660224 +0000 UTC m=+1562.227591420"
Sep 08 14:42:31.462225 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:31.462179    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:42:37.986675 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:37.986644    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:42:47.362593 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:47.362554    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:48.701985 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:48.701944    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:54.039568 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:54.039542    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx_8ca9504a-7456-4cb1-b202-a976227bb406/payload-processing/0.log"
Sep 08 14:42:58.656935 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.656900    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"]
Sep 08 14:42:58.660573 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.660554    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.662928 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.662895    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-274ffa\"/\"kube-root-ca.crt\""
Sep 08 14:42:58.664171 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.664150    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-274ffa\"/\"default-dockercfg-9w2k9\""
Sep 08 14:42:58.664171 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.664174    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-274ffa\"/\"openshift-service-ca.crt\""
Sep 08 14:42:58.664314 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.664151    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-274ffa\"/\"ipp-route-274ffa-kserve-self-signed-certs\""
Sep 08 14:42:58.670909 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.670883    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"]
Sep 08 14:42:58.682099 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.682046    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:58.723676 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.723634    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-dshm\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.723879 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.723693    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-model-cache\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.723879 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.723758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tmp-dir\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.724011 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.723870    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mblp5\" (UniqueName: \"kubernetes.io/projected/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-kube-api-access-mblp5\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.724011 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.723948    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-home\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.724011 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.723967    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tls-certs\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.825137 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.825090    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-model-cache\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.825371 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.825148    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tmp-dir\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.825371 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.825180    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mblp5\" (UniqueName: \"kubernetes.io/projected/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-kube-api-access-mblp5\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.825371 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.825276    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-home\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.825371 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.825298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tls-certs\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.825371 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.825371    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-dshm\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.825654 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.825510    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-model-cache\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.825654 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.825571    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tmp-dir\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.825654 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.825637    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-home\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.827603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.827580    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-dshm\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.827867 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.827849    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tls-certs\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.834780 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.834755    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mblp5\" (UniqueName: \"kubernetes.io/projected/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-kube-api-access-mblp5\") pod \"ipp-route-274ffa-kserve-6f5b5cd7f-776rr\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") " pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:58.972020 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:58.971908    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:42:59.246557 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:42:59.246528    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"]
Sep 08 14:42:59.248943 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:42:59.248907    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfeff52a_6695_4b3b_95be_d3901f6f3c3e.slice/crio-529c236f1880c856a045ee6c7606ff9951a8fb7ff70e50490af71a7029deb95b WatchSource:0}: Error finding container 529c236f1880c856a045ee6c7606ff9951a8fb7ff70e50490af71a7029deb95b: Status 404 returned error can't find the container with id 529c236f1880c856a045ee6c7606ff9951a8fb7ff70e50490af71a7029deb95b
Sep 08 14:43:00.161701 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:00.161662    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr" event={"ID":"dfeff52a-6695-4b3b-95be-d3901f6f3c3e","Type":"ContainerStarted","Data":"50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4"}
Sep 08 14:43:00.161701 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:00.161708    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr" event={"ID":"dfeff52a-6695-4b3b-95be-d3901f6f3c3e","Type":"ContainerStarted","Data":"529c236f1880c856a045ee6c7606ff9951a8fb7ff70e50490af71a7029deb95b"}
Sep 08 14:43:00.179562 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:00.179512    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr" podStartSLOduration=2.179492471 podStartE2EDuration="2.179492471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:42:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:43:00.178122267 +0000 UTC m=+1595.409053467" watchObservedRunningTime="2026-09-08 14:43:00.179492471 +0000 UTC m=+1595.410423710"
Sep 08 14:43:03.895118 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:03.895074    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:04.553745 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:04.553701    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:43:08.972197 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:08.972151    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:43:08.972728 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:08.972235    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:43:08.984827 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:08.984790    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:43:09.220066 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:09.220034    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:43:27.062052 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:27.061964    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:43:32.101977 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:32.101947    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx_8ca9504a-7456-4cb1-b202-a976227bb406/payload-processing/0.log"
Sep 08 14:43:32.380538 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:32.380453    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:32.718827 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:32.718740    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:33.190539 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:33.190505    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:36.101552 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:36.101515    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:36.201635 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:36.201598    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:37.393768 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:37.393725    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:37.460327 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:37.460293    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:37.672061 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:37.671973    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:37.897811 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:37.897780    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:38.000563 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.000528    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:38.082480 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.082446    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"]
Sep 08 14:43:38.086452 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.086431    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:43:38.088939 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.088916    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-6b7995-dockercfg-2nwbp\""
Sep 08 14:43:38.088939 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.088935    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-6b7995\""
Sep 08 14:43:38.096485 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.096456    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"]
Sep 08 14:43:38.211702 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.211652    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abd132b9-027f-4df6-81a8-b9557672ac74-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr\" (UID: \"abd132b9-027f-4df6-81a8-b9557672ac74\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:43:38.211702 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.211702    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6mbf\" (UniqueName: \"kubernetes.io/projected/abd132b9-027f-4df6-81a8-b9557672ac74-kube-api-access-c6mbf\") pod \"payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr\" (UID: \"abd132b9-027f-4df6-81a8-b9557672ac74\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:43:38.312471 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.312379    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abd132b9-027f-4df6-81a8-b9557672ac74-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr\" (UID: \"abd132b9-027f-4df6-81a8-b9557672ac74\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:43:38.312471 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.312422    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6mbf\" (UniqueName: \"kubernetes.io/projected/abd132b9-027f-4df6-81a8-b9557672ac74-kube-api-access-c6mbf\") pod \"payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr\" (UID: \"abd132b9-027f-4df6-81a8-b9557672ac74\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:43:38.313159 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.313137    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abd132b9-027f-4df6-81a8-b9557672ac74-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr\" (UID: \"abd132b9-027f-4df6-81a8-b9557672ac74\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:43:38.321384 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.321354    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6mbf\" (UniqueName: \"kubernetes.io/projected/abd132b9-027f-4df6-81a8-b9557672ac74-kube-api-access-c6mbf\") pod \"payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr\" (UID: \"abd132b9-027f-4df6-81a8-b9557672ac74\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:43:38.398251 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.398190    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:43:38.672457 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:38.672432    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"]
Sep 08 14:43:38.674157 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:43:38.674120    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd132b9_027f_4df6_81a8_b9557672ac74.slice/crio-d8fb55757837f5a0590814a27f7506e7d21362ea97f1038efd54818e62c513d9 WatchSource:0}: Error finding container d8fb55757837f5a0590814a27f7506e7d21362ea97f1038efd54818e62c513d9: Status 404 returned error can't find the container with id d8fb55757837f5a0590814a27f7506e7d21362ea97f1038efd54818e62c513d9
Sep 08 14:43:39.373512 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:39.373473    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr" event={"ID":"abd132b9-027f-4df6-81a8-b9557672ac74","Type":"ContainerStarted","Data":"361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2"}
Sep 08 14:43:39.373741 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:39.373631    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:43:39.373741 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:39.373669    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr" event={"ID":"abd132b9-027f-4df6-81a8-b9557672ac74","Type":"ContainerStarted","Data":"d8fb55757837f5a0590814a27f7506e7d21362ea97f1038efd54818e62c513d9"}
Sep 08 14:43:39.390779 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:39.390729    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr" podStartSLOduration=1.390713946 podStartE2EDuration="1.390713946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:43:39.389402721 +0000 UTC m=+1634.620333918" watchObservedRunningTime="2026-09-08 14:43:39.390713946 +0000 UTC m=+1634.621645143"
Sep 08 14:43:43.848970 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:43.848927    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:43:50.383539 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:43:50.383500    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:44:01.384573 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:01.384539    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:01.836076 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:01.836038    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"]
Sep 08 14:44:01.839773 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:01.839747    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"
Sep 08 14:44:01.842327 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:01.842302    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:44:01.842449 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:01.842355    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fddbb\""
Sep 08 14:44:01.847178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:01.847148    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"]
Sep 08 14:44:01.847609 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:01.847574    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f435ec-9642-4b6f-ade2-f54a3735ced0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc\" (UID: \"d9f435ec-9642-4b6f-ade2-f54a3735ced0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"
Sep 08 14:44:01.948725 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:01.948680    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f435ec-9642-4b6f-ade2-f54a3735ced0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc\" (UID: \"d9f435ec-9642-4b6f-ade2-f54a3735ced0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"
Sep 08 14:44:01.949359 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:01.949340    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f435ec-9642-4b6f-ade2-f54a3735ced0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc\" (UID: \"d9f435ec-9642-4b6f-ade2-f54a3735ced0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"
Sep 08 14:44:02.153606 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:02.153517    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"
Sep 08 14:44:02.424673 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:02.424646    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"]
Sep 08 14:44:02.426474 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:44:02.426445    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f435ec_9642_4b6f_ade2_f54a3735ced0.slice/crio-20c7b90a047c3af9a8095ace0c4d0111655b2a8ddd19be4215e9dafe78f0ad1a WatchSource:0}: Error finding container 20c7b90a047c3af9a8095ace0c4d0111655b2a8ddd19be4215e9dafe78f0ad1a: Status 404 returned error can't find the container with id 20c7b90a047c3af9a8095ace0c4d0111655b2a8ddd19be4215e9dafe78f0ad1a
Sep 08 14:44:02.494629 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:02.494591    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc" event={"ID":"d9f435ec-9642-4b6f-ade2-f54a3735ced0","Type":"ContainerStarted","Data":"20c7b90a047c3af9a8095ace0c4d0111655b2a8ddd19be4215e9dafe78f0ad1a"}
Sep 08 14:44:03.501859 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:03.501822    2558 generic.go:356] "Generic (PLEG): container finished" podID="d9f435ec-9642-4b6f-ade2-f54a3735ced0" containerID="1b1851421a3295c47c9a803e7b9c1cab125eb58f1707ed444def7c423b227eba" exitCode=0
Sep 08 14:44:03.502272 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:03.501862    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc" event={"ID":"d9f435ec-9642-4b6f-ade2-f54a3735ced0","Type":"ContainerDied","Data":"1b1851421a3295c47c9a803e7b9c1cab125eb58f1707ed444def7c423b227eba"}
Sep 08 14:44:04.653499 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:04.653474    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"
Sep 08 14:44:04.678698 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:04.678667    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f435ec-9642-4b6f-ade2-f54a3735ced0-maas-api-service-ca\") pod \"d9f435ec-9642-4b6f-ade2-f54a3735ced0\" (UID: \"d9f435ec-9642-4b6f-ade2-f54a3735ced0\") "
Sep 08 14:44:04.679021 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:04.678994    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9f435ec-9642-4b6f-ade2-f54a3735ced0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d9f435ec-9642-4b6f-ade2-f54a3735ced0" (UID: "d9f435ec-9642-4b6f-ade2-f54a3735ced0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:44:04.780457 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:04.780359    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f435ec-9642-4b6f-ade2-f54a3735ced0-maas-api-service-ca\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:05.518789 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:05.518751    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc" event={"ID":"d9f435ec-9642-4b6f-ade2-f54a3735ced0","Type":"ContainerDied","Data":"20c7b90a047c3af9a8095ace0c4d0111655b2a8ddd19be4215e9dafe78f0ad1a"}
Sep 08 14:44:05.518789 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:05.518776    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"
Sep 08 14:44:05.518996 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:05.518809    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20c7b90a047c3af9a8095ace0c4d0111655b2a8ddd19be4215e9dafe78f0ad1a"
Sep 08 14:44:08.129953 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.129918    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:08.168819 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.168779    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"]
Sep 08 14:44:08.169076 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.169045    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr" podUID="abd132b9-027f-4df6-81a8-b9557672ac74" containerName="payload-pre-processing" containerID="cri-o://361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2" gracePeriod=30
Sep 08 14:44:08.229692 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.229660    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:08.340622 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.340591    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"]
Sep 08 14:44:08.342515 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.342493    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-6b7995-cmcsc"]
Sep 08 14:44:08.422272 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.422243    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:44:08.517334 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.517304    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abd132b9-027f-4df6-81a8-b9557672ac74-plugins-config-volume\") pod \"abd132b9-027f-4df6-81a8-b9557672ac74\" (UID: \"abd132b9-027f-4df6-81a8-b9557672ac74\") "
Sep 08 14:44:08.517518 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.517400    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6mbf\" (UniqueName: \"kubernetes.io/projected/abd132b9-027f-4df6-81a8-b9557672ac74-kube-api-access-c6mbf\") pod \"abd132b9-027f-4df6-81a8-b9557672ac74\" (UID: \"abd132b9-027f-4df6-81a8-b9557672ac74\") "
Sep 08 14:44:08.517797 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.517768    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abd132b9-027f-4df6-81a8-b9557672ac74-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "abd132b9-027f-4df6-81a8-b9557672ac74" (UID: "abd132b9-027f-4df6-81a8-b9557672ac74"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:44:08.519562 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.519534    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abd132b9-027f-4df6-81a8-b9557672ac74-kube-api-access-c6mbf" (OuterVolumeSpecName: "kube-api-access-c6mbf") pod "abd132b9-027f-4df6-81a8-b9557672ac74" (UID: "abd132b9-027f-4df6-81a8-b9557672ac74"). InnerVolumeSpecName "kube-api-access-c6mbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:44:08.536178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.536145    2558 generic.go:356] "Generic (PLEG): container finished" podID="abd132b9-027f-4df6-81a8-b9557672ac74" containerID="361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2" exitCode=0
Sep 08 14:44:08.536351 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.536187    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"
Sep 08 14:44:08.536351 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.536195    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr" event={"ID":"abd132b9-027f-4df6-81a8-b9557672ac74","Type":"ContainerDied","Data":"361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2"}
Sep 08 14:44:08.536351 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.536288    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr" event={"ID":"abd132b9-027f-4df6-81a8-b9557672ac74","Type":"ContainerDied","Data":"d8fb55757837f5a0590814a27f7506e7d21362ea97f1038efd54818e62c513d9"}
Sep 08 14:44:08.536351 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.536319    2558 scope.go:117] "RemoveContainer" containerID="361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2"
Sep 08 14:44:08.545374 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.545351    2558 scope.go:117] "RemoveContainer" containerID="361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2"
Sep 08 14:44:08.545686 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:44:08.545658    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2\": container with ID starting with 361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2 not found: ID does not exist" containerID="361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2"
Sep 08 14:44:08.545753 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.545696    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2"} err="failed to get container status \"361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2\": rpc error: code = NotFound desc = could not find container \"361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2\": container with ID starting with 361e09ee9fe9c6a2bff76ac8a34e9d627f251efb2382f0c8089c22f0c12fc1b2 not found: ID does not exist"
Sep 08 14:44:08.557971 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.557944    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"]
Sep 08 14:44:08.561115 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.561090    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b7995-755c876cd6-cnzzr"]
Sep 08 14:44:08.618617 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.618581    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6mbf\" (UniqueName: \"kubernetes.io/projected/abd132b9-027f-4df6-81a8-b9557672ac74-kube-api-access-c6mbf\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:08.618617 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.618611    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abd132b9-027f-4df6-81a8-b9557672ac74-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:08.677321 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.677246    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:08.824408 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:08.824374    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:09.287047 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:09.287004    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abd132b9-027f-4df6-81a8-b9557672ac74" path="/var/lib/kubelet/pods/abd132b9-027f-4df6-81a8-b9557672ac74/volumes"
Sep 08 14:44:09.287439 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:09.287403    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9f435ec-9642-4b6f-ade2-f54a3735ced0" path="/var/lib/kubelet/pods/d9f435ec-9642-4b6f-ade2-f54a3735ced0/volumes"
Sep 08 14:44:13.955032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:13.954996    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:44:15.063666 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:15.063630    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:15.168226 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:15.165017    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:15.328728 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:15.328632    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:19.138667 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.138628    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"]
Sep 08 14:44:19.139129 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.139110    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abd132b9-027f-4df6-81a8-b9557672ac74" containerName="payload-pre-processing"
Sep 08 14:44:19.139177 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.139133    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="abd132b9-027f-4df6-81a8-b9557672ac74" containerName="payload-pre-processing"
Sep 08 14:44:19.139177 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.139165    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9f435ec-9642-4b6f-ade2-f54a3735ced0" containerName="revoke-keys"
Sep 08 14:44:19.139177 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.139173    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9f435ec-9642-4b6f-ade2-f54a3735ced0" containerName="revoke-keys"
Sep 08 14:44:19.139318 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.139300    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="abd132b9-027f-4df6-81a8-b9557672ac74" containerName="payload-pre-processing"
Sep 08 14:44:19.139363 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.139318    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9f435ec-9642-4b6f-ade2-f54a3735ced0" containerName="revoke-keys"
Sep 08 14:44:19.143800 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.143782    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"
Sep 08 14:44:19.146828 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.146801    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:44:19.148163 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.148097    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fddbb\""
Sep 08 14:44:19.150845 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.150817    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"]
Sep 08 14:44:19.225628 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.225592    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9ed2b1-1439-4443-b841-80fa7ae5a793-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj\" (UID: \"8d9ed2b1-1439-4443-b841-80fa7ae5a793\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"
Sep 08 14:44:19.279041 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.279001    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:19.326508 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.326464    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9ed2b1-1439-4443-b841-80fa7ae5a793-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj\" (UID: \"8d9ed2b1-1439-4443-b841-80fa7ae5a793\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"
Sep 08 14:44:19.327108 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.327086    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9ed2b1-1439-4443-b841-80fa7ae5a793-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj\" (UID: \"8d9ed2b1-1439-4443-b841-80fa7ae5a793\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"
Sep 08 14:44:19.456938 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.456844    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"
Sep 08 14:44:19.726976 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:19.726944    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"]
Sep 08 14:44:19.729914 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:44:19.729880    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d9ed2b1_1439_4443_b841_80fa7ae5a793.slice/crio-d3445705eb36adc28ffc21e1612c8fd9df7aeb8b6d21c1a151d1e4f19b9d10e1 WatchSource:0}: Error finding container d3445705eb36adc28ffc21e1612c8fd9df7aeb8b6d21c1a151d1e4f19b9d10e1: Status 404 returned error can't find the container with id d3445705eb36adc28ffc21e1612c8fd9df7aeb8b6d21c1a151d1e4f19b9d10e1
Sep 08 14:44:20.604634 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:20.604598    2558 generic.go:356] "Generic (PLEG): container finished" podID="8d9ed2b1-1439-4443-b841-80fa7ae5a793" containerID="a34797f437a666bf8c9a95ee66f5780c06cb98ec9f3f626a8ea64b3e32d1d108" exitCode=0
Sep 08 14:44:20.605025 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:20.604693    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj" event={"ID":"8d9ed2b1-1439-4443-b841-80fa7ae5a793","Type":"ContainerDied","Data":"a34797f437a666bf8c9a95ee66f5780c06cb98ec9f3f626a8ea64b3e32d1d108"}
Sep 08 14:44:20.605025 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:20.604759    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj" event={"ID":"8d9ed2b1-1439-4443-b841-80fa7ae5a793","Type":"ContainerStarted","Data":"d3445705eb36adc28ffc21e1612c8fd9df7aeb8b6d21c1a151d1e4f19b9d10e1"}
Sep 08 14:44:21.743022 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:21.742995    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"
Sep 08 14:44:21.851372 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:21.851325    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9ed2b1-1439-4443-b841-80fa7ae5a793-maas-api-service-ca\") pod \"8d9ed2b1-1439-4443-b841-80fa7ae5a793\" (UID: \"8d9ed2b1-1439-4443-b841-80fa7ae5a793\") "
Sep 08 14:44:21.851682 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:21.851657    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d9ed2b1-1439-4443-b841-80fa7ae5a793-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8d9ed2b1-1439-4443-b841-80fa7ae5a793" (UID: "8d9ed2b1-1439-4443-b841-80fa7ae5a793"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:44:21.953129 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:21.953041    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9ed2b1-1439-4443-b841-80fa7ae5a793-maas-api-service-ca\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:22.616423 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:22.616397    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"
Sep 08 14:44:22.616603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:22.616395    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj" event={"ID":"8d9ed2b1-1439-4443-b841-80fa7ae5a793","Type":"ContainerDied","Data":"d3445705eb36adc28ffc21e1612c8fd9df7aeb8b6d21c1a151d1e4f19b9d10e1"}
Sep 08 14:44:22.616603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:22.616521    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3445705eb36adc28ffc21e1612c8fd9df7aeb8b6d21c1a151d1e4f19b9d10e1"
Sep 08 14:44:34.150251 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:34.150195    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:44:34.676991 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:34.676952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:34.721442 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:34.721400    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"]
Sep 08 14:44:34.721662 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:34.721630    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p" podUID="6c821f86-fb94-4c55-8675-53c333662759" containerName="payload-pre-processing" containerID="cri-o://acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111" gracePeriod=30
Sep 08 14:44:34.746408 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:34.746377    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:34.861015 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:34.860986    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:34.954432 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:34.954348    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"]
Sep 08 14:44:34.956477 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:34.956449    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-274ffa-pbgfj"]
Sep 08 14:44:34.981940 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:34.981917    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:44:34.989762 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:34.989731    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:35.081483 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.081450    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcdgg\" (UniqueName: \"kubernetes.io/projected/6c821f86-fb94-4c55-8675-53c333662759-kube-api-access-bcdgg\") pod \"6c821f86-fb94-4c55-8675-53c333662759\" (UID: \"6c821f86-fb94-4c55-8675-53c333662759\") "
Sep 08 14:44:35.081658 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.081495    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c821f86-fb94-4c55-8675-53c333662759-plugins-config-volume\") pod \"6c821f86-fb94-4c55-8675-53c333662759\" (UID: \"6c821f86-fb94-4c55-8675-53c333662759\") "
Sep 08 14:44:35.081939 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.081913    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c821f86-fb94-4c55-8675-53c333662759-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6c821f86-fb94-4c55-8675-53c333662759" (UID: "6c821f86-fb94-4c55-8675-53c333662759"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:44:35.083679 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.083657    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c821f86-fb94-4c55-8675-53c333662759-kube-api-access-bcdgg" (OuterVolumeSpecName: "kube-api-access-bcdgg") pod "6c821f86-fb94-4c55-8675-53c333662759" (UID: "6c821f86-fb94-4c55-8675-53c333662759"). InnerVolumeSpecName "kube-api-access-bcdgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:44:35.182516 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.182472    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcdgg\" (UniqueName: \"kubernetes.io/projected/6c821f86-fb94-4c55-8675-53c333662759-kube-api-access-bcdgg\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:35.182516 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.182509    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c821f86-fb94-4c55-8675-53c333662759-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:35.287404 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.287371    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d9ed2b1-1439-4443-b841-80fa7ae5a793" path="/var/lib/kubelet/pods/8d9ed2b1-1439-4443-b841-80fa7ae5a793/volumes"
Sep 08 14:44:35.340740 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.340709    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:35.681697 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.681611    2558 generic.go:356] "Generic (PLEG): container finished" podID="6c821f86-fb94-4c55-8675-53c333662759" containerID="acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111" exitCode=0
Sep 08 14:44:35.681697 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.681650    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"
Sep 08 14:44:35.681697 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.681651    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p" event={"ID":"6c821f86-fb94-4c55-8675-53c333662759","Type":"ContainerDied","Data":"acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111"}
Sep 08 14:44:35.681956 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.681710    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p" event={"ID":"6c821f86-fb94-4c55-8675-53c333662759","Type":"ContainerDied","Data":"7ebcdfe6e65dd0360220e9342573d5b6ae672083f22db2f41ca84228c51499a9"}
Sep 08 14:44:35.681956 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.681733    2558 scope.go:117] "RemoveContainer" containerID="acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111"
Sep 08 14:44:35.690421 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.690401    2558 scope.go:117] "RemoveContainer" containerID="acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111"
Sep 08 14:44:35.690704 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:44:35.690685    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111\": container with ID starting with acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111 not found: ID does not exist" containerID="acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111"
Sep 08 14:44:35.690760 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.690712    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111"} err="failed to get container status \"acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111\": rpc error: code = NotFound desc = could not find container \"acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111\": container with ID starting with acd968e7f7032fdeac8d0e2287e8a331cbbc6d9a1fd7ebff109a6231a8864111 not found: ID does not exist"
Sep 08 14:44:35.699148 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.699122    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"]
Sep 08 14:44:35.704031 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:35.704009    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-274ffa-86bfc584b6-lbk7p"]
Sep 08 14:44:37.287122 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:37.287082    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c821f86-fb94-4c55-8675-53c333662759" path="/var/lib/kubelet/pods/6c821f86-fb94-4c55-8675-53c333662759/volumes"
Sep 08 14:44:40.322638 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:40.322600    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"]
Sep 08 14:44:40.368848 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:40.368813    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:43.846537 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:43.846501    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:44:55.342613 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.342501    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr" podUID="dfeff52a-6695-4b3b-95be-d3901f6f3c3e" containerName="main" containerID="cri-o://50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4" gracePeriod=45
Sep 08 14:44:55.598912 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.598849    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:44:55.690178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.690147    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mblp5\" (UniqueName: \"kubernetes.io/projected/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-kube-api-access-mblp5\") pod \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") "
Sep 08 14:44:55.690385 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.690200    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tmp-dir\") pod \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") "
Sep 08 14:44:55.690385 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.690264    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tls-certs\") pod \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") "
Sep 08 14:44:55.690385 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.690297    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-dshm\") pod \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") "
Sep 08 14:44:55.690385 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.690358    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-model-cache\") pod \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") "
Sep 08 14:44:55.690385 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.690388    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-home\") pod \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\" (UID: \"dfeff52a-6695-4b3b-95be-d3901f6f3c3e\") "
Sep 08 14:44:55.690665 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.690486    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dfeff52a-6695-4b3b-95be-d3901f6f3c3e" (UID: "dfeff52a-6695-4b3b-95be-d3901f6f3c3e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:44:55.690719 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.690689    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tmp-dir\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:55.690780 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.690724    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-home" (OuterVolumeSpecName: "home") pod "dfeff52a-6695-4b3b-95be-d3901f6f3c3e" (UID: "dfeff52a-6695-4b3b-95be-d3901f6f3c3e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:44:55.690780 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.690740    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-model-cache" (OuterVolumeSpecName: "model-cache") pod "dfeff52a-6695-4b3b-95be-d3901f6f3c3e" (UID: "dfeff52a-6695-4b3b-95be-d3901f6f3c3e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:44:55.692552 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.692526    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-dshm" (OuterVolumeSpecName: "dshm") pod "dfeff52a-6695-4b3b-95be-d3901f6f3c3e" (UID: "dfeff52a-6695-4b3b-95be-d3901f6f3c3e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:44:55.692552 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.692542    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-kube-api-access-mblp5" (OuterVolumeSpecName: "kube-api-access-mblp5") pod "dfeff52a-6695-4b3b-95be-d3901f6f3c3e" (UID: "dfeff52a-6695-4b3b-95be-d3901f6f3c3e"). InnerVolumeSpecName "kube-api-access-mblp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:44:55.692552 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.692531    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dfeff52a-6695-4b3b-95be-d3901f6f3c3e" (UID: "dfeff52a-6695-4b3b-95be-d3901f6f3c3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:44:55.783061 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.783025    2558 generic.go:356] "Generic (PLEG): container finished" podID="dfeff52a-6695-4b3b-95be-d3901f6f3c3e" containerID="50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4" exitCode=0
Sep 08 14:44:55.783281 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.783072    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"
Sep 08 14:44:55.783281 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.783085    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr" event={"ID":"dfeff52a-6695-4b3b-95be-d3901f6f3c3e","Type":"ContainerDied","Data":"50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4"}
Sep 08 14:44:55.783281 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.783117    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr" event={"ID":"dfeff52a-6695-4b3b-95be-d3901f6f3c3e","Type":"ContainerDied","Data":"529c236f1880c856a045ee6c7606ff9951a8fb7ff70e50490af71a7029deb95b"}
Sep 08 14:44:55.783281 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.783136    2558 scope.go:117] "RemoveContainer" containerID="50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4"
Sep 08 14:44:55.791571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.791544    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-tls-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:55.791571 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.791573    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-dshm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:55.791715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.791582    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-model-cache\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:55.791715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.791590    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-home\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:55.791715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.791599    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mblp5\" (UniqueName: \"kubernetes.io/projected/dfeff52a-6695-4b3b-95be-d3901f6f3c3e-kube-api-access-mblp5\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:44:55.792731 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.792714    2558 scope.go:117] "RemoveContainer" containerID="50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4"
Sep 08 14:44:55.792998 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:44:55.792982    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4\": container with ID starting with 50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4 not found: ID does not exist" containerID="50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4"
Sep 08 14:44:55.793049 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.793005    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4"} err="failed to get container status \"50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4\": rpc error: code = NotFound desc = could not find container \"50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4\": container with ID starting with 50637e1eb171ccc536cf7ce97b20b041ac02d59438dafe49143676747658a7a4 not found: ID does not exist"
Sep 08 14:44:55.806644 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.806614    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"]
Sep 08 14:44:55.812246 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:55.812218    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-274ffa/ipp-route-274ffa-kserve-6f5b5cd7f-776rr"]
Sep 08 14:44:57.288221 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:44:57.288179    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfeff52a-6695-4b3b-95be-d3901f6f3c3e" path="/var/lib/kubelet/pods/dfeff52a-6695-4b3b-95be-d3901f6f3c3e/volumes"
Sep 08 14:45:00.147665 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.147621    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj"]
Sep 08 14:45:00.148121 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.148026    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d9ed2b1-1439-4443-b841-80fa7ae5a793" containerName="revoke-keys"
Sep 08 14:45:00.148121 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.148038    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d9ed2b1-1439-4443-b841-80fa7ae5a793" containerName="revoke-keys"
Sep 08 14:45:00.148121 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.148049    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c821f86-fb94-4c55-8675-53c333662759" containerName="payload-pre-processing"
Sep 08 14:45:00.148121 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.148054    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c821f86-fb94-4c55-8675-53c333662759" containerName="payload-pre-processing"
Sep 08 14:45:00.148121 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.148076    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfeff52a-6695-4b3b-95be-d3901f6f3c3e" containerName="main"
Sep 08 14:45:00.148121 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.148081    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfeff52a-6695-4b3b-95be-d3901f6f3c3e" containerName="main"
Sep 08 14:45:00.148488 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.148138    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c821f86-fb94-4c55-8675-53c333662759" containerName="payload-pre-processing"
Sep 08 14:45:00.148488 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.148146    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfeff52a-6695-4b3b-95be-d3901f6f3c3e" containerName="main"
Sep 08 14:45:00.148488 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.148153    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d9ed2b1-1439-4443-b841-80fa7ae5a793" containerName="revoke-keys"
Sep 08 14:45:00.151268 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.151247    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69"]
Sep 08 14:45:00.151399 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.151376    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj"
Sep 08 14:45:00.154448 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.154429    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fddbb\""
Sep 08 14:45:00.154624 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.154607    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69"
Sep 08 14:45:00.163501 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.163478    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69"]
Sep 08 14:45:00.163888 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.163868    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj"
Sep 08 14:45:00.165577 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.165557    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj"]
Sep 08 14:45:00.172447 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.172427    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69"
Sep 08 14:45:00.444235 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.444174    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj"]
Sep 08 14:45:00.445951 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:45:00.445920    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podede16345_d8dc_4811_b48d_61573ae305fa.slice/crio-aa13db42ea71c92f9b9b2cd4099a1329fbfd1d4f5630eda4487c4d41bed1b294 WatchSource:0}: Error finding container aa13db42ea71c92f9b9b2cd4099a1329fbfd1d4f5630eda4487c4d41bed1b294: Status 404 returned error can't find the container with id aa13db42ea71c92f9b9b2cd4099a1329fbfd1d4f5630eda4487c4d41bed1b294
Sep 08 14:45:00.469146 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.469115    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69"]
Sep 08 14:45:00.472425 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:45:00.472401    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05cf1906_d2d2_4f87_ae70_08a4339992c3.slice/crio-5fdc165e89fdd5f56fc8208d9ceb525039b98dba3bed7f07bc54d52480c93035 WatchSource:0}: Error finding container 5fdc165e89fdd5f56fc8208d9ceb525039b98dba3bed7f07bc54d52480c93035: Status 404 returned error can't find the container with id 5fdc165e89fdd5f56fc8208d9ceb525039b98dba3bed7f07bc54d52480c93035
Sep 08 14:45:00.811588 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.811545    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69" event={"ID":"05cf1906-d2d2-4f87-ae70-08a4339992c3","Type":"ContainerStarted","Data":"5fdc165e89fdd5f56fc8208d9ceb525039b98dba3bed7f07bc54d52480c93035"}
Sep 08 14:45:00.813342 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.813314    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj" event={"ID":"ede16345-d8dc-4811-b48d-61573ae305fa","Type":"ContainerStarted","Data":"8bba2e2540fd406f84e2fb3352da574215a35b7b50412358c552d4605179f507"}
Sep 08 14:45:00.813478 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.813398    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj" event={"ID":"ede16345-d8dc-4811-b48d-61573ae305fa","Type":"ContainerStarted","Data":"aa13db42ea71c92f9b9b2cd4099a1329fbfd1d4f5630eda4487c4d41bed1b294"}
Sep 08 14:45:00.834431 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:00.834381    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj" podStartSLOduration=0.834367688 podStartE2EDuration="834.367688ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:45:00.833872512 +0000 UTC m=+1716.064803710" watchObservedRunningTime="2026-09-08 14:45:00.834367688 +0000 UTC m=+1716.065298914"
Sep 08 14:45:01.820045 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:01.820010    2558 generic.go:356] "Generic (PLEG): container finished" podID="05cf1906-d2d2-4f87-ae70-08a4339992c3" containerID="2e43efe86cf17607d94b58847a709604c6585eedb2cf856c17d3d7d730af2a35" exitCode=0
Sep 08 14:45:01.820488 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:01.820097    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69" event={"ID":"05cf1906-d2d2-4f87-ae70-08a4339992c3","Type":"ContainerDied","Data":"2e43efe86cf17607d94b58847a709604c6585eedb2cf856c17d3d7d730af2a35"}
Sep 08 14:45:01.821715 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:01.821693    2558 generic.go:356] "Generic (PLEG): container finished" podID="ede16345-d8dc-4811-b48d-61573ae305fa" containerID="8bba2e2540fd406f84e2fb3352da574215a35b7b50412358c552d4605179f507" exitCode=0
Sep 08 14:45:01.821836 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:01.821731    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj" event={"ID":"ede16345-d8dc-4811-b48d-61573ae305fa","Type":"ContainerDied","Data":"8bba2e2540fd406f84e2fb3352da574215a35b7b50412358c552d4605179f507"}
Sep 08 14:45:02.975013 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:02.974977    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj"
Sep 08 14:45:02.978283 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:02.978259    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69"
Sep 08 14:45:03.833500 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:03.833470    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69"
Sep 08 14:45:03.833703 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:03.833502    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69" event={"ID":"05cf1906-d2d2-4f87-ae70-08a4339992c3","Type":"ContainerDied","Data":"5fdc165e89fdd5f56fc8208d9ceb525039b98dba3bed7f07bc54d52480c93035"}
Sep 08 14:45:03.833703 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:03.833547    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fdc165e89fdd5f56fc8208d9ceb525039b98dba3bed7f07bc54d52480c93035"
Sep 08 14:45:03.834921 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:03.834900    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj" event={"ID":"ede16345-d8dc-4811-b48d-61573ae305fa","Type":"ContainerDied","Data":"aa13db42ea71c92f9b9b2cd4099a1329fbfd1d4f5630eda4487c4d41bed1b294"}
Sep 08 14:45:03.834921 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:03.834922    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj"
Sep 08 14:45:03.835075 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:03.834932    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa13db42ea71c92f9b9b2cd4099a1329fbfd1d4f5630eda4487c4d41bed1b294"
Sep 08 14:45:15.713861 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:15.713823    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:15.772690 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:15.772655    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:16.561115 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:16.561075    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:18.037546 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:18.037510    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:23.884590 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:23.884557    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:23.966673 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:23.966638    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:24.186880 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:24.186789    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"]
Sep 08 14:45:24.187048 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:24.186999    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy" containerID="cri-o://80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74" gracePeriod=30
Sep 08 14:45:24.211030 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:24.210998    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:24.963795 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:24.963762    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:29.433298 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.433274    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:45:29.539382 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539348    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-socket\") pod \"14f24731-0f42-4a1e-8d8f-d58131befb0e\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") "
Sep 08 14:45:29.539566 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539403    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14f24731-0f42-4a1e-8d8f-d58131befb0e-istiod-ca-cert\") pod \"14f24731-0f42-4a1e-8d8f-d58131befb0e\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") "
Sep 08 14:45:29.539566 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539421    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrfgm\" (UniqueName: \"kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-kube-api-access-jrfgm\") pod \"14f24731-0f42-4a1e-8d8f-d58131befb0e\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") "
Sep 08 14:45:29.539566 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539450    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-certs\") pod \"14f24731-0f42-4a1e-8d8f-d58131befb0e\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") "
Sep 08 14:45:29.539566 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539484    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-envoy\") pod \"14f24731-0f42-4a1e-8d8f-d58131befb0e\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") "
Sep 08 14:45:29.539566 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539564    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-token\") pod \"14f24731-0f42-4a1e-8d8f-d58131befb0e\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") "
Sep 08 14:45:29.539841 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539604    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-data\") pod \"14f24731-0f42-4a1e-8d8f-d58131befb0e\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") "
Sep 08 14:45:29.539841 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539646    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-podinfo\") pod \"14f24731-0f42-4a1e-8d8f-d58131befb0e\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") "
Sep 08 14:45:29.539841 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539656    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "14f24731-0f42-4a1e-8d8f-d58131befb0e" (UID: "14f24731-0f42-4a1e-8d8f-d58131befb0e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:45:29.539841 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539666    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-credential-socket\") pod \"14f24731-0f42-4a1e-8d8f-d58131befb0e\" (UID: \"14f24731-0f42-4a1e-8d8f-d58131befb0e\") "
Sep 08 14:45:29.539841 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539820    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "14f24731-0f42-4a1e-8d8f-d58131befb0e" (UID: "14f24731-0f42-4a1e-8d8f-d58131befb0e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:45:29.540067 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539907    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "14f24731-0f42-4a1e-8d8f-d58131befb0e" (UID: "14f24731-0f42-4a1e-8d8f-d58131befb0e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:45:29.540067 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.539936    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14f24731-0f42-4a1e-8d8f-d58131befb0e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "14f24731-0f42-4a1e-8d8f-d58131befb0e" (UID: "14f24731-0f42-4a1e-8d8f-d58131befb0e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:45:29.540067 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.540030    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-data" (OuterVolumeSpecName: "istio-data") pod "14f24731-0f42-4a1e-8d8f-d58131befb0e" (UID: "14f24731-0f42-4a1e-8d8f-d58131befb0e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:45:29.540267 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.540246    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-data\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:29.540342 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.540272    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-credential-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:29.540342 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.540288    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:29.540342 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.540302    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14f24731-0f42-4a1e-8d8f-d58131befb0e-istiod-ca-cert\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:29.540342 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.540314    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-workload-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:29.541708 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.541684    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-kube-api-access-jrfgm" (OuterVolumeSpecName: "kube-api-access-jrfgm") pod "14f24731-0f42-4a1e-8d8f-d58131befb0e" (UID: "14f24731-0f42-4a1e-8d8f-d58131befb0e"). InnerVolumeSpecName "kube-api-access-jrfgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:45:29.541810 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.541708    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-token" (OuterVolumeSpecName: "istio-token") pod "14f24731-0f42-4a1e-8d8f-d58131befb0e" (UID: "14f24731-0f42-4a1e-8d8f-d58131befb0e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:45:29.542080 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.542055    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "14f24731-0f42-4a1e-8d8f-d58131befb0e" (UID: "14f24731-0f42-4a1e-8d8f-d58131befb0e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:45:29.542080 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.542061    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "14f24731-0f42-4a1e-8d8f-d58131befb0e" (UID: "14f24731-0f42-4a1e-8d8f-d58131befb0e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:45:29.641089 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.641049    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrfgm\" (UniqueName: \"kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-kube-api-access-jrfgm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:29.641089 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.641084    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-envoy\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:29.641089 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.641094    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-token\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:29.641374 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.641103    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14f24731-0f42-4a1e-8d8f-d58131befb0e-istio-podinfo\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:29.969230 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.969124    2558 generic.go:356] "Generic (PLEG): container finished" podID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerID="80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74" exitCode=0
Sep 08 14:45:29.969230 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.969168    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"
Sep 08 14:45:29.969434 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.969230    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" event={"ID":"14f24731-0f42-4a1e-8d8f-d58131befb0e","Type":"ContainerDied","Data":"80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74"}
Sep 08 14:45:29.969434 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.969290    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b" event={"ID":"14f24731-0f42-4a1e-8d8f-d58131befb0e","Type":"ContainerDied","Data":"9eae57afa3a4e440e52939b08cea18d01a81bfdeabeeb99bf21a8e9303c4c9dd"}
Sep 08 14:45:29.969434 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.969316    2558 scope.go:117] "RemoveContainer" containerID="80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74"
Sep 08 14:45:29.978759 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.978741    2558 scope.go:117] "RemoveContainer" containerID="113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64"
Sep 08 14:45:29.986746 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.986730    2558 scope.go:117] "RemoveContainer" containerID="80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74"
Sep 08 14:45:29.986999 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:45:29.986981    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74\": container with ID starting with 80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74 not found: ID does not exist" containerID="80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74"
Sep 08 14:45:29.987043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.987009    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74"} err="failed to get container status \"80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74\": rpc error: code = NotFound desc = could not find container \"80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74\": container with ID starting with 80d5369d6d34c1c952e37ff9535b485cdd218dcddb9b333885b1f12b9b386c74 not found: ID does not exist"
Sep 08 14:45:29.987043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.987025    2558 scope.go:117] "RemoveContainer" containerID="113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64"
Sep 08 14:45:29.987282 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:45:29.987264    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64\": container with ID starting with 113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64 not found: ID does not exist" containerID="113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64"
Sep 08 14:45:29.987348 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:29.987286    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64"} err="failed to get container status \"113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64\": rpc error: code = NotFound desc = could not find container \"113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64\": container with ID starting with 113b28f8cc74841b3d854bdd8b54a2e89855a8c50c8b80a9a474b89f12e13d64 not found: ID does not exist"
Sep 08 14:45:30.002655 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:30.002624    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"]
Sep 08 14:45:30.012296 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:30.012270    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-274ffa-openshift-default-759c4766b-thc7b"]
Sep 08 14:45:31.287275 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:31.287244    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" path="/var/lib/kubelet/pods/14f24731-0f42-4a1e-8d8f-d58131befb0e/volumes"
Sep 08 14:45:35.281467 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.281428    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:35.286903 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.286876    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj"]
Sep 08 14:45:35.292392 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.292365    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f7d00a-29814645-r4txj"]
Sep 08 14:45:35.322781 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.322740    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"]
Sep 08 14:45:35.323072 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.323014    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx" podUID="8ca9504a-7456-4cb1-b202-a976227bb406" containerName="payload-processing" containerID="cri-o://03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e" gracePeriod=30
Sep 08 14:45:35.352124 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.352082    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:35.561931 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.561819    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:35.577535 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.577511    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:45:35.623192 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.623163    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:35.698234 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.698191    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca9504a-7456-4cb1-b202-a976227bb406-plugins-config-volume\") pod \"8ca9504a-7456-4cb1-b202-a976227bb406\" (UID: \"8ca9504a-7456-4cb1-b202-a976227bb406\") "
Sep 08 14:45:35.698410 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.698331    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hdpv\" (UniqueName: \"kubernetes.io/projected/8ca9504a-7456-4cb1-b202-a976227bb406-kube-api-access-9hdpv\") pod \"8ca9504a-7456-4cb1-b202-a976227bb406\" (UID: \"8ca9504a-7456-4cb1-b202-a976227bb406\") "
Sep 08 14:45:35.698600 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.698576    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ca9504a-7456-4cb1-b202-a976227bb406-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ca9504a-7456-4cb1-b202-a976227bb406" (UID: "8ca9504a-7456-4cb1-b202-a976227bb406"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:45:35.700509 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.700484    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ca9504a-7456-4cb1-b202-a976227bb406-kube-api-access-9hdpv" (OuterVolumeSpecName: "kube-api-access-9hdpv") pod "8ca9504a-7456-4cb1-b202-a976227bb406" (UID: "8ca9504a-7456-4cb1-b202-a976227bb406"). InnerVolumeSpecName "kube-api-access-9hdpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:45:35.799467 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.799418    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca9504a-7456-4cb1-b202-a976227bb406-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:35.799467 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:35.799462    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hdpv\" (UniqueName: \"kubernetes.io/projected/8ca9504a-7456-4cb1-b202-a976227bb406-kube-api-access-9hdpv\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:36.001150 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:36.001116    2558 generic.go:356] "Generic (PLEG): container finished" podID="8ca9504a-7456-4cb1-b202-a976227bb406" containerID="03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e" exitCode=0
Sep 08 14:45:36.001360 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:36.001161    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"
Sep 08 14:45:36.001360 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:36.001228    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx" event={"ID":"8ca9504a-7456-4cb1-b202-a976227bb406","Type":"ContainerDied","Data":"03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e"}
Sep 08 14:45:36.001360 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:36.001286    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx" event={"ID":"8ca9504a-7456-4cb1-b202-a976227bb406","Type":"ContainerDied","Data":"d75be741fe84d18ac3b6f4d1906c2c9850e0c929189a43d67f350249daed5b99"}
Sep 08 14:45:36.001360 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:36.001315    2558 scope.go:117] "RemoveContainer" containerID="03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e"
Sep 08 14:45:36.010752 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:36.010734    2558 scope.go:117] "RemoveContainer" containerID="03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e"
Sep 08 14:45:36.011029 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:45:36.011013    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e\": container with ID starting with 03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e not found: ID does not exist" containerID="03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e"
Sep 08 14:45:36.011074 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:36.011036    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e"} err="failed to get container status \"03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e\": rpc error: code = NotFound desc = could not find container \"03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e\": container with ID starting with 03376c8dd077f1e3862b45ea5797560fd6e8cb13c8abb220cc70fd19b31dc71e not found: ID does not exist"
Sep 08 14:45:36.024155 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:36.024129    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"]
Sep 08 14:45:36.031553 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:36.031528    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f7d00a-5794fd4b7-gsstx"]
Sep 08 14:45:36.065990 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:36.065965    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:37.287325 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:37.287292    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ca9504a-7456-4cb1-b202-a976227bb406" path="/var/lib/kubelet/pods/8ca9504a-7456-4cb1-b202-a976227bb406/volumes"
Sep 08 14:45:37.287741 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:37.287611    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ede16345-d8dc-4811-b48d-61573ae305fa" path="/var/lib/kubelet/pods/ede16345-d8dc-4811-b48d-61573ae305fa/volumes"
Sep 08 14:45:38.153114 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:38.153071    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:45:42.603550 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:42.603512    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:42.702657 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:42.702612    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:42.890584 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:42.890473    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:43.264637 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.264603    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"]
Sep 08 14:45:43.264978 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.264966    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy"
Sep 08 14:45:43.265032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.264979    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy"
Sep 08 14:45:43.265032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.264987    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy"
Sep 08 14:45:43.265032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.264992    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy"
Sep 08 14:45:43.265032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.265006    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05cf1906-d2d2-4f87-ae70-08a4339992c3" containerName="cleanup"
Sep 08 14:45:43.265032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.265013    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="05cf1906-d2d2-4f87-ae70-08a4339992c3" containerName="cleanup"
Sep 08 14:45:43.265032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.265024    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ede16345-d8dc-4811-b48d-61573ae305fa" containerName="cleanup"
Sep 08 14:45:43.265032 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.265029    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ede16345-d8dc-4811-b48d-61573ae305fa" containerName="cleanup"
Sep 08 14:45:43.265288 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.265042    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca9504a-7456-4cb1-b202-a976227bb406" containerName="payload-processing"
Sep 08 14:45:43.265288 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.265047    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca9504a-7456-4cb1-b202-a976227bb406" containerName="payload-processing"
Sep 08 14:45:43.265288 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.265110    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy"
Sep 08 14:45:43.265288 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.265117    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="14f24731-0f42-4a1e-8d8f-d58131befb0e" containerName="istio-proxy"
Sep 08 14:45:43.265288 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.265124    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ede16345-d8dc-4811-b48d-61573ae305fa" containerName="cleanup"
Sep 08 14:45:43.265288 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.265130    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="05cf1906-d2d2-4f87-ae70-08a4339992c3" containerName="cleanup"
Sep 08 14:45:43.265288 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.265138    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ca9504a-7456-4cb1-b202-a976227bb406" containerName="payload-processing"
Sep 08 14:45:43.269563 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.269544    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"
Sep 08 14:45:43.271976 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.271948    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:45:43.272117 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.271988    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fddbb\""
Sep 08 14:45:43.275748 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.275724    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"]
Sep 08 14:45:43.370392 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.370357    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/213f3330-b6e9-4206-8ddc-8df4f9945a7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc\" (UID: \"213f3330-b6e9-4206-8ddc-8df4f9945a7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"
Sep 08 14:45:43.471089 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.471050    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/213f3330-b6e9-4206-8ddc-8df4f9945a7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc\" (UID: \"213f3330-b6e9-4206-8ddc-8df4f9945a7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"
Sep 08 14:45:43.471674 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.471653    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/213f3330-b6e9-4206-8ddc-8df4f9945a7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc\" (UID: \"213f3330-b6e9-4206-8ddc-8df4f9945a7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"
Sep 08 14:45:43.482453 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.482420    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:43.580935 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.580836    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"
Sep 08 14:45:43.855504 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:43.855478    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"]
Sep 08 14:45:43.857640 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:45:43.857613    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213f3330_b6e9_4206_8ddc_8df4f9945a7f.slice/crio-a4777e6b4dd208d015c0ee42c7018f99553d29c30489f326271bb214425ad1df WatchSource:0}: Error finding container a4777e6b4dd208d015c0ee42c7018f99553d29c30489f326271bb214425ad1df: Status 404 returned error can't find the container with id a4777e6b4dd208d015c0ee42c7018f99553d29c30489f326271bb214425ad1df
Sep 08 14:45:44.044537 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:44.044494    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc" event={"ID":"213f3330-b6e9-4206-8ddc-8df4f9945a7f","Type":"ContainerStarted","Data":"a4777e6b4dd208d015c0ee42c7018f99553d29c30489f326271bb214425ad1df"}
Sep 08 14:45:45.051584 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:45.051547    2558 generic.go:356] "Generic (PLEG): container finished" podID="213f3330-b6e9-4206-8ddc-8df4f9945a7f" containerID="2a501b3fc47cbeb1af89d86edbc3ff7648459c4b5b886430c2add537e4161fbb" exitCode=0
Sep 08 14:45:45.052017 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:45.051630    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc" event={"ID":"213f3330-b6e9-4206-8ddc-8df4f9945a7f","Type":"ContainerDied","Data":"2a501b3fc47cbeb1af89d86edbc3ff7648459c4b5b886430c2add537e4161fbb"}
Sep 08 14:45:46.188802 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:46.188776    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"
Sep 08 14:45:46.195770 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:46.195748    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/213f3330-b6e9-4206-8ddc-8df4f9945a7f-maas-api-service-ca\") pod \"213f3330-b6e9-4206-8ddc-8df4f9945a7f\" (UID: \"213f3330-b6e9-4206-8ddc-8df4f9945a7f\") "
Sep 08 14:45:46.196049 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:46.196027    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/213f3330-b6e9-4206-8ddc-8df4f9945a7f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "213f3330-b6e9-4206-8ddc-8df4f9945a7f" (UID: "213f3330-b6e9-4206-8ddc-8df4f9945a7f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:45:46.297179 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:46.297146    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/213f3330-b6e9-4206-8ddc-8df4f9945a7f-maas-api-service-ca\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:45:47.062921 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:47.062891    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"
Sep 08 14:45:47.063238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:47.062937    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc" event={"ID":"213f3330-b6e9-4206-8ddc-8df4f9945a7f","Type":"ContainerDied","Data":"a4777e6b4dd208d015c0ee42c7018f99553d29c30489f326271bb214425ad1df"}
Sep 08 14:45:47.063238 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:47.062982    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4777e6b4dd208d015c0ee42c7018f99553d29c30489f326271bb214425ad1df"
Sep 08 14:45:53.820245 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:53.820193    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:53.902526 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:53.902492    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:54.003743 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:54.003711    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:54.064424 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:54.064393    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:54.079057 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:54.078969    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"]
Sep 08 14:45:54.080876 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:54.080855    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-780944-q7lcc"]
Sep 08 14:45:54.172254 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:54.172220    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:54.600870 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:54.600833    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:55.288292 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:55.288256    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="213f3330-b6e9-4206-8ddc-8df4f9945a7f" path="/var/lib/kubelet/pods/213f3330-b6e9-4206-8ddc-8df4f9945a7f/volumes"
Sep 08 14:45:56.551638 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:45:56.551603    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:46:00.975639 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:00.975601    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:01.082474 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:01.082438    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:01.483127 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:01.483091    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:01.968567 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:01.968485    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:12.381768 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.381736    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:12.387107 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.387077    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69"]
Sep 08 14:46:12.389549 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.389527    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-017dbd-29814645-mpb69"]
Sep 08 14:46:12.432302 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.432268    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"]
Sep 08 14:46:12.432512 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.432478    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb" podUID="8fd21f5f-823a-4057-ac83-34865aaa3d1c" containerName="payload-pre-processing" containerID="cri-o://62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0" gracePeriod=30
Sep 08 14:46:12.459107 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.459079    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:12.522878 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.522842    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:12.625585 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.625555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:12.681833 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.681799    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:12.687382 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.687357    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:46:12.853852 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.853814    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvcqs\" (UniqueName: \"kubernetes.io/projected/8fd21f5f-823a-4057-ac83-34865aaa3d1c-kube-api-access-lvcqs\") pod \"8fd21f5f-823a-4057-ac83-34865aaa3d1c\" (UID: \"8fd21f5f-823a-4057-ac83-34865aaa3d1c\") "
Sep 08 14:46:12.854027 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.853883    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fd21f5f-823a-4057-ac83-34865aaa3d1c-plugins-config-volume\") pod \"8fd21f5f-823a-4057-ac83-34865aaa3d1c\" (UID: \"8fd21f5f-823a-4057-ac83-34865aaa3d1c\") "
Sep 08 14:46:12.854324 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.854301    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fd21f5f-823a-4057-ac83-34865aaa3d1c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8fd21f5f-823a-4057-ac83-34865aaa3d1c" (UID: "8fd21f5f-823a-4057-ac83-34865aaa3d1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:46:12.856030 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.856010    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fd21f5f-823a-4057-ac83-34865aaa3d1c-kube-api-access-lvcqs" (OuterVolumeSpecName: "kube-api-access-lvcqs") pod "8fd21f5f-823a-4057-ac83-34865aaa3d1c" (UID: "8fd21f5f-823a-4057-ac83-34865aaa3d1c"). InnerVolumeSpecName "kube-api-access-lvcqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:46:12.955595 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.955511    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvcqs\" (UniqueName: \"kubernetes.io/projected/8fd21f5f-823a-4057-ac83-34865aaa3d1c-kube-api-access-lvcqs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:46:12.955595 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:12.955543    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fd21f5f-823a-4057-ac83-34865aaa3d1c-plugins-config-volume\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:46:13.018285 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.018248    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:13.195911 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.195875    2558 generic.go:356] "Generic (PLEG): container finished" podID="8fd21f5f-823a-4057-ac83-34865aaa3d1c" containerID="62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0" exitCode=0
Sep 08 14:46:13.196088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.195924    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"
Sep 08 14:46:13.196088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.195962    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb" event={"ID":"8fd21f5f-823a-4057-ac83-34865aaa3d1c","Type":"ContainerDied","Data":"62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0"}
Sep 08 14:46:13.196088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.196016    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb" event={"ID":"8fd21f5f-823a-4057-ac83-34865aaa3d1c","Type":"ContainerDied","Data":"54fa182e92bd89ece1dc327a978388373c0790bf31eb3bd623fe0f7ca7e9e9df"}
Sep 08 14:46:13.196088 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.196039    2558 scope.go:117] "RemoveContainer" containerID="62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0"
Sep 08 14:46:13.205000 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.204982    2558 scope.go:117] "RemoveContainer" containerID="62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0"
Sep 08 14:46:13.205311 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:46:13.205290    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0\": container with ID starting with 62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0 not found: ID does not exist" containerID="62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0"
Sep 08 14:46:13.205395 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.205320    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0"} err="failed to get container status \"62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0\": rpc error: code = NotFound desc = could not find container \"62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0\": container with ID starting with 62e1839424f1823d7c673399c471892ccd184f0d291d1f06dac0439e54d6eac0 not found: ID does not exist"
Sep 08 14:46:13.221043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.220987    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"]
Sep 08 14:46:13.228150 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.228130    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-017dbd-5c78fdd946-s6pdb"]
Sep 08 14:46:13.287821 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.287778    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05cf1906-d2d2-4f87-ae70-08a4339992c3" path="/var/lib/kubelet/pods/05cf1906-d2d2-4f87-ae70-08a4339992c3/volumes"
Sep 08 14:46:13.288388 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:13.288364    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fd21f5f-823a-4057-ac83-34865aaa3d1c" path="/var/lib/kubelet/pods/8fd21f5f-823a-4057-ac83-34865aaa3d1c/volumes"
Sep 08 14:46:14.960056 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:14.960024    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:46:19.680084 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:19.679978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:19.739781 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:19.739739    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:19.938022 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:19.937926    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"]
Sep 08 14:46:19.938198 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:19.938129    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy" containerID="cri-o://67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c" gracePeriod=30
Sep 08 14:46:19.961900 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:19.961871    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:23.337911 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:23.337872    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 14:46:23.338326 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:23.337940    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 14:46:23.750570 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:23.750539    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:46:25.192114 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.192086    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:46:25.260010 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.259982    2558 generic.go:356] "Generic (PLEG): container finished" podID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerID="67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c" exitCode=0
Sep 08 14:46:25.260178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.260028    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"
Sep 08 14:46:25.260178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.260075    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" event={"ID":"519c6d5f-819a-469f-ab5f-267ac1d7cd7d","Type":"ContainerDied","Data":"67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c"}
Sep 08 14:46:25.260178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.260131    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt" event={"ID":"519c6d5f-819a-469f-ab5f-267ac1d7cd7d","Type":"ContainerDied","Data":"1434f423f5c150c310a63468a86d59156645de37b2f827330b332ecad7e48cf5"}
Sep 08 14:46:25.260178 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.260154    2558 scope.go:117] "RemoveContainer" containerID="67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c"
Sep 08 14:46:25.268943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.268927    2558 scope.go:117] "RemoveContainer" containerID="1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8"
Sep 08 14:46:25.269008 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.268988    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-certs\") pod \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") "
Sep 08 14:46:25.269065 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269051    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istiod-ca-cert\") pod \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") "
Sep 08 14:46:25.269121 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269097    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-envoy\") pod \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") "
Sep 08 14:46:25.269121 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269116    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-credential-socket\") pod \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") "
Sep 08 14:46:25.269250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269148    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-podinfo\") pod \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") "
Sep 08 14:46:25.269250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269178    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-data\") pod \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") "
Sep 08 14:46:25.269250 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269230    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-token\") pod \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") "
Sep 08 14:46:25.269454 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269258    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxvfd\" (UniqueName: \"kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-kube-api-access-wxvfd\") pod \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") "
Sep 08 14:46:25.269454 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269292    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-socket\") pod \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\" (UID: \"519c6d5f-819a-469f-ab5f-267ac1d7cd7d\") "
Sep 08 14:46:25.269454 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269322    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "519c6d5f-819a-469f-ab5f-267ac1d7cd7d" (UID: "519c6d5f-819a-469f-ab5f-267ac1d7cd7d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:46:25.269603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269513    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "519c6d5f-819a-469f-ab5f-267ac1d7cd7d" (UID: "519c6d5f-819a-469f-ab5f-267ac1d7cd7d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:46:25.269603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269559    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "519c6d5f-819a-469f-ab5f-267ac1d7cd7d" (UID: "519c6d5f-819a-469f-ab5f-267ac1d7cd7d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:46:25.269713 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269652    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:46:25.269713 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269671    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-workload-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:46:25.269713 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269687    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istiod-ca-cert\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:46:25.269874 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269771    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "519c6d5f-819a-469f-ab5f-267ac1d7cd7d" (UID: "519c6d5f-819a-469f-ab5f-267ac1d7cd7d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:46:25.269874 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.269786    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-data" (OuterVolumeSpecName: "istio-data") pod "519c6d5f-819a-469f-ab5f-267ac1d7cd7d" (UID: "519c6d5f-819a-469f-ab5f-267ac1d7cd7d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:46:25.271697 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.271672    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "519c6d5f-819a-469f-ab5f-267ac1d7cd7d" (UID: "519c6d5f-819a-469f-ab5f-267ac1d7cd7d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:46:25.271866 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.271757    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-kube-api-access-wxvfd" (OuterVolumeSpecName: "kube-api-access-wxvfd") pod "519c6d5f-819a-469f-ab5f-267ac1d7cd7d" (UID: "519c6d5f-819a-469f-ab5f-267ac1d7cd7d"). InnerVolumeSpecName "kube-api-access-wxvfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:46:25.271941 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.271906    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "519c6d5f-819a-469f-ab5f-267ac1d7cd7d" (UID: "519c6d5f-819a-469f-ab5f-267ac1d7cd7d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:46:25.271941 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.271921    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-token" (OuterVolumeSpecName: "istio-token") pod "519c6d5f-819a-469f-ab5f-267ac1d7cd7d" (UID: "519c6d5f-819a-469f-ab5f-267ac1d7cd7d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:46:25.293829 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.293801    2558 scope.go:117] "RemoveContainer" containerID="67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c"
Sep 08 14:46:25.294336 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:46:25.294310    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c\": container with ID starting with 67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c not found: ID does not exist" containerID="67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c"
Sep 08 14:46:25.294439 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.294345    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c"} err="failed to get container status \"67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c\": rpc error: code = NotFound desc = could not find container \"67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c\": container with ID starting with 67c787f3846733743fbfde503cef6a48d7eed4e036c0d90878d104609ada787c not found: ID does not exist"
Sep 08 14:46:25.294439 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.294364    2558 scope.go:117] "RemoveContainer" containerID="1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8"
Sep 08 14:46:25.294678 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:46:25.294659    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8\": container with ID starting with 1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8 not found: ID does not exist" containerID="1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8"
Sep 08 14:46:25.294721 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.294685    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8"} err="failed to get container status \"1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8\": rpc error: code = NotFound desc = could not find container \"1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8\": container with ID starting with 1261e66c1e5a9e8b7479ce4a29b6d272e909e64235322a2dfad411fc96a7b7f8 not found: ID does not exist"
Sep 08 14:46:25.370311 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.370283    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-envoy\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:46:25.370311 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.370307    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-credential-socket\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:46:25.370311 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.370318    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-podinfo\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:46:25.370586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.370326    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-data\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:46:25.370586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.370334    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-istio-token\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:46:25.370586 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.370341    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxvfd\" (UniqueName: \"kubernetes.io/projected/519c6d5f-819a-469f-ab5f-267ac1d7cd7d-kube-api-access-wxvfd\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:46:25.577920 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.577835    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"]
Sep 08 14:46:25.584070 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:25.584049    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-017dbd-openshift-default-75487bf846-7pmtt"]
Sep 08 14:46:27.287993 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:46:27.287955    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" path="/var/lib/kubelet/pods/519c6d5f-819a-469f-ab5f-267ac1d7cd7d/volumes"
Sep 08 14:47:43.596964 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:47:43.596922    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:44.750318 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:47:44.750281    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:47:45.198459 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:47:45.198363    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:46.005791 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:47:46.005744    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:10.950603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:48:10.950566    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:48:18.960094 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:48:18.960056    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:48:25.608661 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:48:25.608627    2558 scope.go:117] "RemoveContainer" containerID="f038d6e340ec02dcd6e296bec62dd18454cfe143d72da9d4b4dd01bc91fc11f2"
Sep 08 14:48:28.749464 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:48:28.749429    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:48:44.361105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:48:44.361068    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:48:48.565804 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:48:48.565755    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:56.152865 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:48:56.152823    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:49:11.373978 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:49:11.373947    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:20.746158 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:20.746120    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:50:25.635569 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:25.635535    2558 scope.go:117] "RemoveContainer" containerID="a34797f437a666bf8c9a95ee66f5780c06cb98ec9f3f626a8ea64b3e32d1d108"
Sep 08 14:50:25.643973 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:25.643953    2558 scope.go:117] "RemoveContainer" containerID="1b1851421a3295c47c9a803e7b9c1cab125eb58f1707ed444def7c423b227eba"
Sep 08 14:50:27.452368 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:27.452337    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:50:34.446220 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.446170    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:50:34.877822 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.877782    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"]
Sep 08 14:50:34.878273 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878252    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fd21f5f-823a-4057-ac83-34865aaa3d1c" containerName="payload-pre-processing"
Sep 08 14:50:34.878380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878275    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fd21f5f-823a-4057-ac83-34865aaa3d1c" containerName="payload-pre-processing"
Sep 08 14:50:34.878380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878288    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy"
Sep 08 14:50:34.878380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878296    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy"
Sep 08 14:50:34.878380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878312    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy"
Sep 08 14:50:34.878380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878321    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy"
Sep 08 14:50:34.878380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878341    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="213f3330-b6e9-4206-8ddc-8df4f9945a7f" containerName="revoke-keys"
Sep 08 14:50:34.878380 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878349    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="213f3330-b6e9-4206-8ddc-8df4f9945a7f" containerName="revoke-keys"
Sep 08 14:50:34.878720 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878467    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy"
Sep 08 14:50:34.878720 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878480    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="519c6d5f-819a-469f-ab5f-267ac1d7cd7d" containerName="istio-proxy"
Sep 08 14:50:34.878720 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878491    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fd21f5f-823a-4057-ac83-34865aaa3d1c" containerName="payload-pre-processing"
Sep 08 14:50:34.878720 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.878499    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="213f3330-b6e9-4206-8ddc-8df4f9945a7f" containerName="revoke-keys"
Sep 08 14:50:34.881533 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.881510    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:34.885007 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.884988    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-35588716-kserve-self-signed-certs\""
Sep 08 14:50:34.898942 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.898918    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"]
Sep 08 14:50:34.923844 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.923816    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:34.960748 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.960722    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9f1fecc-40b4-45d1-836f-6100e043f03e-tls-certs\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:34.960913 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.960758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-tmp-dir\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:34.960913 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.960779    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-dshm\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:34.960913 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.960863    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-home\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:34.961080 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.960945    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-model-cache\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:34.961080 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:34.960985    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm5mr\" (UniqueName: \"kubernetes.io/projected/f9f1fecc-40b4-45d1-836f-6100e043f03e-kube-api-access-nm5mr\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.062092 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.062057    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9f1fecc-40b4-45d1-836f-6100e043f03e-tls-certs\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.062286 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.062100    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-tmp-dir\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.062286 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.062117    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-dshm\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.062286 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.062251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-home\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.062457 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.062330    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-model-cache\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.062457 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.062363    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm5mr\" (UniqueName: \"kubernetes.io/projected/f9f1fecc-40b4-45d1-836f-6100e043f03e-kube-api-access-nm5mr\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.062618 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.062589    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-tmp-dir\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.062736 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.062678    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-home\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.062736 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.062726    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-model-cache\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.064420 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.064403    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-dshm\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.064672 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.064655    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9f1fecc-40b4-45d1-836f-6100e043f03e-tls-certs\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.070703 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.070685    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm5mr\" (UniqueName: \"kubernetes.io/projected/f9f1fecc-40b4-45d1-836f-6100e043f03e-kube-api-access-nm5mr\") pod \"e2e-same-id-a-35588716-kserve-7588667df5-bxj5l\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") " pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.190902 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.190819    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:35.462083 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.462017    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"]
Sep 08 14:50:35.464427 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:50:35.464401    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9f1fecc_40b4_45d1_836f_6100e043f03e.slice/crio-76bfb9ed3b56d435eced8b0f55b6b0ed58b5908b1bf051111cadb33529b3ee44 WatchSource:0}: Error finding container 76bfb9ed3b56d435eced8b0f55b6b0ed58b5908b1bf051111cadb33529b3ee44: Status 404 returned error can't find the container with id 76bfb9ed3b56d435eced8b0f55b6b0ed58b5908b1bf051111cadb33529b3ee44
Sep 08 14:50:35.483933 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:35.483898    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l" event={"ID":"f9f1fecc-40b4-45d1-836f-6100e043f03e","Type":"ContainerStarted","Data":"76bfb9ed3b56d435eced8b0f55b6b0ed58b5908b1bf051111cadb33529b3ee44"}
Sep 08 14:50:36.494062 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:36.494025    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l" event={"ID":"f9f1fecc-40b4-45d1-836f-6100e043f03e","Type":"ContainerStarted","Data":"19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68"}
Sep 08 14:50:36.512014 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:36.511961    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l" podStartSLOduration=2.511944836 podStartE2EDuration="2.511944836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:50:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:50:36.50940889 +0000 UTC m=+2051.740340086" watchObservedRunningTime="2026-09-08 14:50:36.511944836 +0000 UTC m=+2051.742876033"
Sep 08 14:50:42.046705 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:42.046673    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:50:44.143940 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:44.143905    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:44.202788 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:44.202759    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:44.717060 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:44.717026    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:44.772686 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:44.772652    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:45.191730 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:45.191688    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:45.192314 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:45.191761    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:45.204406 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:45.204380    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:45.549832 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:45.549800    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:50:46.681086 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:46.681054    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:50.658141 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:50:50.658054    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:51:12.838000 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:12.837959    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"]
Sep 08 14:51:12.870164 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:12.870133    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:14.658943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:14.658911    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:51:14.696781 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:14.696748    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:23.946378 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:23.946339    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:51:25.668176 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:25.668140    2558 scope.go:117] "RemoveContainer" containerID="2e43efe86cf17607d94b58847a709604c6585eedb2cf856c17d3d7d730af2a35"
Sep 08 14:51:25.676230 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:25.676195    2558 scope.go:117] "RemoveContainer" containerID="8bba2e2540fd406f84e2fb3352da574215a35b7b50412358c552d4605179f507"
Sep 08 14:51:27.857677 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:27.857631    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l" podUID="f9f1fecc-40b4-45d1-836f-6100e043f03e" containerName="main" containerID="cri-o://19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68" gracePeriod=45
Sep 08 14:51:28.106942 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.106915    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:51:28.163448 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.163358    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-model-cache\") pod \"f9f1fecc-40b4-45d1-836f-6100e043f03e\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") "
Sep 08 14:51:28.163448 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.163399    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-tmp-dir\") pod \"f9f1fecc-40b4-45d1-836f-6100e043f03e\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") "
Sep 08 14:51:28.163448 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.163443    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm5mr\" (UniqueName: \"kubernetes.io/projected/f9f1fecc-40b4-45d1-836f-6100e043f03e-kube-api-access-nm5mr\") pod \"f9f1fecc-40b4-45d1-836f-6100e043f03e\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") "
Sep 08 14:51:28.163710 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.163484    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9f1fecc-40b4-45d1-836f-6100e043f03e-tls-certs\") pod \"f9f1fecc-40b4-45d1-836f-6100e043f03e\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") "
Sep 08 14:51:28.163710 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.163535    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-dshm\") pod \"f9f1fecc-40b4-45d1-836f-6100e043f03e\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") "
Sep 08 14:51:28.163710 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.163567    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-home\") pod \"f9f1fecc-40b4-45d1-836f-6100e043f03e\" (UID: \"f9f1fecc-40b4-45d1-836f-6100e043f03e\") "
Sep 08 14:51:28.163710 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.163617    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-model-cache" (OuterVolumeSpecName: "model-cache") pod "f9f1fecc-40b4-45d1-836f-6100e043f03e" (UID: "f9f1fecc-40b4-45d1-836f-6100e043f03e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:51:28.163710 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.163682    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f9f1fecc-40b4-45d1-836f-6100e043f03e" (UID: "f9f1fecc-40b4-45d1-836f-6100e043f03e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:51:28.163951 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.163911    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-tmp-dir\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:51:28.163951 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.163928    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-model-cache\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:51:28.164054 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.164022    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-home" (OuterVolumeSpecName: "home") pod "f9f1fecc-40b4-45d1-836f-6100e043f03e" (UID: "f9f1fecc-40b4-45d1-836f-6100e043f03e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:51:28.165752 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.165728    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9f1fecc-40b4-45d1-836f-6100e043f03e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f9f1fecc-40b4-45d1-836f-6100e043f03e" (UID: "f9f1fecc-40b4-45d1-836f-6100e043f03e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:51:28.165976 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.165950    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f1fecc-40b4-45d1-836f-6100e043f03e-kube-api-access-nm5mr" (OuterVolumeSpecName: "kube-api-access-nm5mr") pod "f9f1fecc-40b4-45d1-836f-6100e043f03e" (UID: "f9f1fecc-40b4-45d1-836f-6100e043f03e"). InnerVolumeSpecName "kube-api-access-nm5mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:51:28.166072 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.166003    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-dshm" (OuterVolumeSpecName: "dshm") pod "f9f1fecc-40b4-45d1-836f-6100e043f03e" (UID: "f9f1fecc-40b4-45d1-836f-6100e043f03e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:51:28.264386 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.264353    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm5mr\" (UniqueName: \"kubernetes.io/projected/f9f1fecc-40b4-45d1-836f-6100e043f03e-kube-api-access-nm5mr\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:51:28.264386 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.264381    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9f1fecc-40b4-45d1-836f-6100e043f03e-tls-certs\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:51:28.264386 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.264391    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-dshm\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:51:28.264631 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.264399    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9f1fecc-40b4-45d1-836f-6100e043f03e-home\") on node \"ip-10-0-131-22.ec2.internal\" DevicePath \"\""
Sep 08 14:51:28.743218 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.743184    2558 generic.go:356] "Generic (PLEG): container finished" podID="f9f1fecc-40b4-45d1-836f-6100e043f03e" containerID="19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68" exitCode=0
Sep 08 14:51:28.743404 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.743269    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"
Sep 08 14:51:28.743404 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.743276    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l" event={"ID":"f9f1fecc-40b4-45d1-836f-6100e043f03e","Type":"ContainerDied","Data":"19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68"}
Sep 08 14:51:28.743404 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.743323    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l" event={"ID":"f9f1fecc-40b4-45d1-836f-6100e043f03e","Type":"ContainerDied","Data":"76bfb9ed3b56d435eced8b0f55b6b0ed58b5908b1bf051111cadb33529b3ee44"}
Sep 08 14:51:28.743404 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.743340    2558 scope.go:117] "RemoveContainer" containerID="19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68"
Sep 08 14:51:28.754924 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.754905    2558 scope.go:117] "RemoveContainer" containerID="19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68"
Sep 08 14:51:28.755247 ip-10-0-131-22 kubenswrapper[2558]: E0908 14:51:28.755227    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68\": container with ID starting with 19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68 not found: ID does not exist" containerID="19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68"
Sep 08 14:51:28.755340 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.755258    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68"} err="failed to get container status \"19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68\": rpc error: code = NotFound desc = could not find container \"19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68\": container with ID starting with 19a5110511667a1977c3b24a2ec399db0884ba9ea61ed6416e98d82d0d29ac68 not found: ID does not exist"
Sep 08 14:51:28.768557 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.768535    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"]
Sep 08 14:51:28.774987 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:28.774963    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-35588716-kserve-7588667df5-bxj5l"]
Sep 08 14:51:29.287010 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:29.286977    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9f1fecc-40b4-45d1-836f-6100e043f03e" path="/var/lib/kubelet/pods/f9f1fecc-40b4-45d1-836f-6100e043f03e/volumes"
Sep 08 14:51:34.852086 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:34.852042    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:51:41.047351 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:41.047319    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:51:51.053593 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:51.053557    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:51:56.856400 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:51:56.856362    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:52:03.656043 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:03.656003    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-pvltk"]
Sep 08 14:52:13.307231 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:13.307187    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kwhrf_e4bb59a3-1c4c-427f-9bea-3a5179e00133/manager/0.log"
Sep 08 14:52:13.694346 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:13.694271    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-566d57d45b-wzqgs_6254bbfb-4506-4cad-ba3e-aa970ec3bd4e/manager/0.log"
Sep 08 14:52:13.702029 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:13.702008    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-566d57d45b-wzqgs_6254bbfb-4506-4cad-ba3e-aa970ec3bd4e/copy-manifests/0.log"
Sep 08 14:52:15.751105 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:15.751075    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-zzqr9_b431fb06-c0af-416a-99e7-bde37580901e/kuadrant-console-plugin/0.log"
Sep 08 14:52:16.065098 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:16.065015    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-pvltk_0f718b8a-b756-48e4-81f0-75041f5a3211/limitador/0.log"
Sep 08 14:52:16.818173 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:16.818144    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5df94dbdbc-kpft5_59758a79-a64a-4717-905c-88d7f6751099/kube-auth-proxy/0.log"
Sep 08 14:52:17.445128 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:17.445096    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-585d697d6-dm2q6_595b339a-11fd-45dc-a950-63dcd5236b02/router/0.log"
Sep 08 14:52:17.835943 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:17.835912    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-nbfxn_9d16d9e0-a2fe-4d90-810d-22a85e7e3b85/main/0.log"
Sep 08 14:52:18.219040 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:18.218961    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-74k5g_78f55cf2-a241-4538-a5f4-ef105e68cdd2/main/0.log"
Sep 08 14:52:18.468840 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:18.468808    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-pjpg9_381d79dc-fc22-415e-862c-9b252e3f2484/main/0.log"
Sep 08 14:52:18.587836 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:18.587809    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-grbh6_dd4c4ed5-9c56-4596-91fc-97e9a6e5f38e/main/0.log"
Sep 08 14:52:24.158738 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:24.158697    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:24.254863 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:24.254827    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:25.697433 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:25.697398    2558 scope.go:117] "RemoveContainer" containerID="2a501b3fc47cbeb1af89d86edbc3ff7648459c4b5b886430c2add537e4161fbb"
Sep 08 14:52:29.679908 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:29.679868    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-drbnw_4e20eb2e-4a15-409a-9192-aafeec619b23/global-pull-secret-syncer/0.log"
Sep 08 14:52:29.810306 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:29.810276    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log"
Sep 08 14:52:29.932161 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:29.932069    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-22.ec2.internal_aef2fcf38a14b047ca1c2b9fe741b9ea/haproxy/0.log"
Sep 08 14:52:34.543462 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:34.543430    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-zzqr9_b431fb06-c0af-416a-99e7-bde37580901e/kuadrant-console-plugin/0.log"
Sep 08 14:52:34.625936 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:34.625850    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-pvltk_0f718b8a-b756-48e4-81f0-75041f5a3211/limitador/0.log"
Sep 08 14:52:36.414190 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:36.414163    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58c3f155-9e8a-47b2-9723-4f95610a26ef/alertmanager/0.log"
Sep 08 14:52:36.457175 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:36.457153    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58c3f155-9e8a-47b2-9723-4f95610a26ef/config-reloader/0.log"
Sep 08 14:52:36.484042 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:36.484021    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58c3f155-9e8a-47b2-9723-4f95610a26ef/kube-rbac-proxy-web/0.log"
Sep 08 14:52:36.508220 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:36.508185    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58c3f155-9e8a-47b2-9723-4f95610a26ef/kube-rbac-proxy/0.log"
Sep 08 14:52:36.532035 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:36.532007    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58c3f155-9e8a-47b2-9723-4f95610a26ef/kube-rbac-proxy-metric/0.log"
Sep 08 14:52:36.555609 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:36.555585    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58c3f155-9e8a-47b2-9723-4f95610a26ef/prom-label-proxy/0.log"
Sep 08 14:52:36.579964 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:36.579899    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58c3f155-9e8a-47b2-9723-4f95610a26ef/init-config-reloader/0.log"
Sep 08 14:52:36.803055 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:36.803014    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fknj7_a907b903-a15e-487c-87ed-e2852618c431/node-exporter/0.log"
Sep 08 14:52:36.826336 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:36.826309    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fknj7_a907b903-a15e-487c-87ed-e2852618c431/kube-rbac-proxy/0.log"
Sep 08 14:52:36.852862 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:36.852794    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fknj7_a907b903-a15e-487c-87ed-e2852618c431/init-textfile/0.log"
Sep 08 14:52:37.197873 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.197847    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d18b6f3d-28cf-44dc-8e35-d4405e34ba30/prometheus/0.log"
Sep 08 14:52:37.227536 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.227513    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d18b6f3d-28cf-44dc-8e35-d4405e34ba30/config-reloader/0.log"
Sep 08 14:52:37.256295 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.256268    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d18b6f3d-28cf-44dc-8e35-d4405e34ba30/thanos-sidecar/0.log"
Sep 08 14:52:37.287047 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.287020    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d18b6f3d-28cf-44dc-8e35-d4405e34ba30/kube-rbac-proxy-web/0.log"
Sep 08 14:52:37.319896 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.319870    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d18b6f3d-28cf-44dc-8e35-d4405e34ba30/kube-rbac-proxy/0.log"
Sep 08 14:52:37.353906 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.353875    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d18b6f3d-28cf-44dc-8e35-d4405e34ba30/kube-rbac-proxy-thanos/0.log"
Sep 08 14:52:37.380005 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.379939    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d18b6f3d-28cf-44dc-8e35-d4405e34ba30/init-config-reloader/0.log"
Sep 08 14:52:37.606588 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.606561    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65964bbcd4-2xhm5_cd2dbb97-1f84-4321-bb36-54d8f9e79767/thanos-query/0.log"
Sep 08 14:52:37.631990 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.631918    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65964bbcd4-2xhm5_cd2dbb97-1f84-4321-bb36-54d8f9e79767/kube-rbac-proxy-web/0.log"
Sep 08 14:52:37.659515 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.659491    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65964bbcd4-2xhm5_cd2dbb97-1f84-4321-bb36-54d8f9e79767/kube-rbac-proxy/0.log"
Sep 08 14:52:37.683369 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.683348    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65964bbcd4-2xhm5_cd2dbb97-1f84-4321-bb36-54d8f9e79767/prom-label-proxy/0.log"
Sep 08 14:52:37.706665 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.706648    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65964bbcd4-2xhm5_cd2dbb97-1f84-4321-bb36-54d8f9e79767/kube-rbac-proxy-rules/0.log"
Sep 08 14:52:37.734045 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:37.734017    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65964bbcd4-2xhm5_cd2dbb97-1f84-4321-bb36-54d8f9e79767/kube-rbac-proxy-metrics/0.log"
Sep 08 14:52:40.424971 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.424943    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-76785bcfcc-lnsvb_fcf7f7f5-5fdf-4eab-a001-9a70b2927928/console/0.log"
Sep 08 14:52:40.580161 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.580123    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"]
Sep 08 14:52:40.580589 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.580568    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9f1fecc-40b4-45d1-836f-6100e043f03e" containerName="main"
Sep 08 14:52:40.580589 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.580588    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9f1fecc-40b4-45d1-836f-6100e043f03e" containerName="main"
Sep 08 14:52:40.580779 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.580670    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9f1fecc-40b4-45d1-836f-6100e043f03e" containerName="main"
Sep 08 14:52:40.583555 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.583539    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.585942 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.585917    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rt5bz\"/\"kube-root-ca.crt\""
Sep 08 14:52:40.586053 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.586004    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rt5bz\"/\"default-dockercfg-fn8ct\""
Sep 08 14:52:40.586053 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.586030    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rt5bz\"/\"openshift-service-ca.crt\""
Sep 08 14:52:40.592157 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.592133    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"]
Sep 08 14:52:40.637027 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.636976    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-proc\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.637256 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.637044    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9s94\" (UniqueName: \"kubernetes.io/projected/539b1d61-4221-49f6-bf7b-7e9a76e743f6-kube-api-access-k9s94\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.637256 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.637100    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-podres\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.637256 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.637131    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-sys\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.637256 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.637155    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-lib-modules\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.738192 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.738154    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-proc\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.738392 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.738225    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9s94\" (UniqueName: \"kubernetes.io/projected/539b1d61-4221-49f6-bf7b-7e9a76e743f6-kube-api-access-k9s94\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.738392 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.738263    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-podres\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.738392 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.738290    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-sys\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.738392 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.738315    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-lib-modules\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.738392 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.738325    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-sys\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.738392 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.738290    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-proc\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.738635 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.738408    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-podres\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.738635 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.738454    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/539b1d61-4221-49f6-bf7b-7e9a76e743f6-lib-modules\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.746795 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.746771    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9s94\" (UniqueName: \"kubernetes.io/projected/539b1d61-4221-49f6-bf7b-7e9a76e743f6-kube-api-access-k9s94\") pod \"perf-node-gather-daemonset-6fq55\" (UID: \"539b1d61-4221-49f6-bf7b-7e9a76e743f6\") " pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:40.894496 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:40.894456    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:41.165694 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:41.165668    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"]
Sep 08 14:52:41.167125 ip-10-0-131-22 kubenswrapper[2558]: W0908 14:52:41.167101    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod539b1d61_4221_49f6_bf7b_7e9a76e743f6.slice/crio-8dd9d1146f15ed4acb19d195b2ab5987fb58f442d013f5b9979ffaf1ebb934c6 WatchSource:0}: Error finding container 8dd9d1146f15ed4acb19d195b2ab5987fb58f442d013f5b9979ffaf1ebb934c6: Status 404 returned error can't find the container with id 8dd9d1146f15ed4acb19d195b2ab5987fb58f442d013f5b9979ffaf1ebb934c6
Sep 08 14:52:42.080774 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:42.080732    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55" event={"ID":"539b1d61-4221-49f6-bf7b-7e9a76e743f6","Type":"ContainerStarted","Data":"3958d6fe0cacd02e14f83614e677d937693c3f863375f37c814c23692f2ea8fa"}
Sep 08 14:52:42.081232 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:42.080809    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55" event={"ID":"539b1d61-4221-49f6-bf7b-7e9a76e743f6","Type":"ContainerStarted","Data":"8dd9d1146f15ed4acb19d195b2ab5987fb58f442d013f5b9979ffaf1ebb934c6"}
Sep 08 14:52:42.081232 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:42.080847    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:42.097546 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:42.097502    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55" podStartSLOduration=2.097488201 podStartE2EDuration="2.097488201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:52:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:52:42.095974678 +0000 UTC m=+2177.326905876" watchObservedRunningTime="2026-09-08 14:52:42.097488201 +0000 UTC m=+2177.328419398"
Sep 08 14:52:42.513417 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:42.513392    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zz6lc_ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1/dns/0.log"
Sep 08 14:52:42.536264 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:42.536238    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zz6lc_ccc6dc26-8c51-4dac-93a2-8b5a2924c5d1/kube-rbac-proxy/0.log"
Sep 08 14:52:42.587619 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:42.587588    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-shfmt_b23c51e8-9fb2-4e5c-883a-65b643ed3ef2/dns-node-resolver/0.log"
Sep 08 14:52:43.206473 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:43.206446    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7c4rl_859d3474-55bb-4163-b637-cf8a13e219d4/node-ca/0.log"
Sep 08 14:52:44.773657 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:44.773628    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5df94dbdbc-kpft5_59758a79-a64a-4717-905c-88d7f6751099/kube-auth-proxy/0.log"
Sep 08 14:52:44.958725 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:44.958698    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-585d697d6-dm2q6_595b339a-11fd-45dc-a950-63dcd5236b02/router/0.log"
Sep 08 14:52:45.841876 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:45.841850    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gxn5d_a1d1aacf-d0b7-4da2-bd12-beaefcc94966/serve-healthcheck-canary/0.log"
Sep 08 14:52:46.547984 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:46.547953    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z6jv7_98688ac0-dc18-43ff-a88e-83560f34c188/kube-rbac-proxy/0.log"
Sep 08 14:52:46.572089 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:46.572061    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z6jv7_98688ac0-dc18-43ff-a88e-83560f34c188/exporter/0.log"
Sep 08 14:52:46.596530 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:46.596505    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z6jv7_98688ac0-dc18-43ff-a88e-83560f34c188/extractor/0.log"
Sep 08 14:52:48.095262 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:48.095231    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rt5bz/perf-node-gather-daemonset-6fq55"
Sep 08 14:52:49.180418 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:49.180373    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kwhrf_e4bb59a3-1c4c-427f-9bea-3a5179e00133/manager/0.log"
Sep 08 14:52:49.367603 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:49.367574    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-566d57d45b-wzqgs_6254bbfb-4506-4cad-ba3e-aa970ec3bd4e/manager/0.log"
Sep 08 14:52:49.385918 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:49.385895    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-566d57d45b-wzqgs_6254bbfb-4506-4cad-ba3e-aa970ec3bd4e/copy-manifests/0.log"
Sep 08 14:52:50.717359 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:50.717329    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-655586cd5f-w824w_86970df8-d442-47d7-ba84-d7a7c84054bc/manager/0.log"
Sep 08 14:52:50.977424 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:50.977346    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5df94dbdbc-kpft5_59758a79-a64a-4717-905c-88d7f6751099/kube-auth-proxy/0.log"
Sep 08 14:52:51.113740 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:51.113701    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-585d697d6-dm2q6_595b339a-11fd-45dc-a950-63dcd5236b02/router/0.log"
Sep 08 14:52:53.296751 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:53.296679    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5df94dbdbc-kpft5_59758a79-a64a-4717-905c-88d7f6751099/kube-auth-proxy/0.log"
Sep 08 14:52:53.434379 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:53.434352    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-585d697d6-dm2q6_595b339a-11fd-45dc-a950-63dcd5236b02/router/0.log"
Sep 08 14:52:56.458245 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:56.458199    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-cxl6r_d3039d4d-4454-4649-8e1a-6dc0f5bfe51b/kube-storage-version-migrator-operator/1.log"
Sep 08 14:52:56.458984 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:56.458968    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-cxl6r_d3039d4d-4454-4649-8e1a-6dc0f5bfe51b/kube-storage-version-migrator-operator/0.log"
Sep 08 14:52:56.685596 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:56.685561    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:57.624916 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:57.624894    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cqkch_0f419cc2-7833-4240-8628-3a0356e4f630/kube-multus-additional-cni-plugins/0.log"
Sep 08 14:52:57.647916 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:57.647887    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cqkch_0f419cc2-7833-4240-8628-3a0356e4f630/egress-router-binary-copy/0.log"
Sep 08 14:52:57.672577 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:57.672558    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cqkch_0f419cc2-7833-4240-8628-3a0356e4f630/cni-plugins/0.log"
Sep 08 14:52:57.697673 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:57.697654    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cqkch_0f419cc2-7833-4240-8628-3a0356e4f630/bond-cni-plugin/0.log"
Sep 08 14:52:57.720392 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:57.720368    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cqkch_0f419cc2-7833-4240-8628-3a0356e4f630/routeoverride-cni/0.log"
Sep 08 14:52:57.743557 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:57.743533    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cqkch_0f419cc2-7833-4240-8628-3a0356e4f630/whereabouts-cni-bincopy/0.log"
Sep 08 14:52:57.765477 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:57.765456    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cqkch_0f419cc2-7833-4240-8628-3a0356e4f630/whereabouts-cni/0.log"
Sep 08 14:52:57.984162 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:57.984134    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h2sbv_b7cb6c46-c03d-4d43-b7ba-36dfacd49074/kube-multus/0.log"
Sep 08 14:52:58.143717 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:58.143681    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zcpxc_d96b871d-efd7-43e9-b268-978c310e94b4/network-metrics-daemon/0.log"
Sep 08 14:52:58.163905 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:58.163880    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zcpxc_d96b871d-efd7-43e9-b268-978c310e94b4/kube-rbac-proxy/0.log"
Sep 08 14:52:58.449535 ip-10-0-131-22 kubenswrapper[2558]: I0908 14:52:58.449454    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-88pdh_9efb8551-2b48-4721-8ac3-3b02ca1aa047/konnectivity-agent/0.log" limit=1024